Lua vs. XML for data storage

Thank you for your answers so far! I'll take the liberty of summing up the points for future reference.

Drawbacks of using Lua to store data, compared to XML

1. Security both when transferring data and when storing it, especially when receiving data from an unknown source
2. Portability, and ease of access to data by other tools
3. Less useful existing tools like XML Schema validators
4. Less support for parsing the data
5. Problems with circular references
6. Less restrictive -- harder to enforce proper conventions, usage and design

Benefits of using Lua to store data, compared to XML

1. Using a single language for both scripting and data (no need for separate files, no need for special cases when loading)
2. Less code and dependencies because of the usage of a single language
3. More verbose and (arguably) more human readable
4. A lot more flexible, and extensible (it's a programming language after all)
5. Data is "executed" and can be accessed when needed as it sits in the virtual machine
6. Takes less space, especially if compiled to bytecode

If I missed something in the first list, please point it out!

This might not be the kind of answer you expected, but it might help you make your decision.

Blizzard (WoW) uses XML to define UI. It's kinda like XAML in C#, just a lot less powerful and most addons just use XML to bootstrap the addon and then build UI in lua code.

Also WoW actually stores addon "Saved Variables" in .lua files.

In my opinion it doesn't mater that much. Choose something you like and which is easy to use for those who are going to extend your engine.

The good thing about XML is that there are A LOT of tools and code already written to test, write and parse XML which means it could save you some time. For example XML Schema's are very useful for validating user written files (security is just a side effect, the good thing is that if it passes your schema, the data is most likely 100% safe and ready to be plugged into your engine) and there quite a few validators already written for you to use.

Then again some users are scared from XML files (even though they are very readable, maybe too readable) and would prefer something "simpler". If it's just for storage (not configuration) then no one is going to edit those file anyway in most cases. XML will also take more space then lua var dump (shouldn't matter, unless you have a lot data).

I don't think you can go wrong here. Blizzard is using lua for storage and I quite like like how it works.

Lua is a major win for storing data. Convenient, fast, and easily convertible to other formats at need. (Convertibility assumes that your data is representable in other formats, which if it is representable in XML, it will be.)

What are the drawbacks of using Lua as a data storage language?

I'm aware of two drawbacks, the significance of which depends on your application:

• If you have Lua tables that contain circular references, e.g., t1.next == t2 and t2.prev = t1, then the process of writing your Lua structures to disk becomes tedious, and the resulting Lua is harder to read than a simple return <list of big expressions here>. (This has never happened to me, and if your data is representable in XML, it will not happen to you.)

• If you have a lot of data—for example if you are writing an inverted index of the entire Inform 7 manual—then you can run afoul of Lua's limits on the number of distinct constants that can appear in a block. You then have to split things up into multiple blocks or functions, or resort to other solutions. This problem has bitten me, and it's a huge pain in the ass. I would like to write a general solution, but I'm not 100% solid I understand the constraints, and I haven't tackled it yet.

  If your data has fewer than 100,000 numeric and string literals, you don't have to worry about this problem.