I think that there are really two issues here.
I'll attack the second one first. You didn't say which version of Acrobat you are using, but if you have any version of Acrobat prior to Acrobat 11, fully updated, and are on a HiDPI Mac (i.e., the so-called Retina screens), that messy display of Audit Space Usage may be due to that. It has nothing to do with the content of your PDF file.
However, what is interesting is that the Audit Space Usage shows the category Document Overhead with nearly 86% of the file space. What Document Overhead represents is a combination of document metadata and private data from other applications (including the private data from Illustrator when you place .AI files or .PDF files from Illustrator in which you did not disable the save editability feature). The document metadata can include XMP data in placed images but the major component of that metadata is XMP metadata that InDesign adds to your document every time you open and edit your document. When you do a simple save of your document, new metadata gets appended. And all that metadata is exported when you export PDF. When you invoked the PDF optimizer in Acrobat, what really slimmed down your PDF file was the option Discard document information and metadata!
Solution without having to invoke Acrobat's optimization feature?
Try saving your InDesign documents using Save as as opposed to simple Save. That should strip out much of the metadata. If that doesn't clean it up significantly, try saving as .IDML and then opening the .IDML file and resaving as .INDD.
By the way, this issue is nothing new to InDesign 10. It is actually less problematic in recent versions than in previous ones.
Let us know if this helps.
- Dov