Common Navigation File

[majestic]

The name duplication of unique items in the books seems to require manual intervention of some sort. At least I can't think of an easy way to do this with a parser.

[alderaine]

jsager - could you draw up the XML reading code to get all the information needed from the CURRENT Aon XML for a given page? Essentially, all the code needed to go to the AON website, load the XML, find the page number, and extract all the available information.
This would be extremely helpful - certainly to me, and probably to others.
The language should not matter too much, as the general syntax will be similar for most.

[alderaine]

In general, book readers with potential access to the internet should attempt to read the web versions first.
Prior to the XML upgrade, this is probably easier with the HTML version, but once we update the XML everything should be there.
Offline readers will always have to have some option to download/ship the books, but there should be a "refresh from the web" option where possible.

[alderaine]

lacerated - good observation
It is not just the name duplication - about 70% of the book content (at a guess) needs manual intervention (hence the toolkit idea) - very little is available in the text at the moment that can be automated.
jsager is right - once we have complete XML, we *should* be able to publish a core engine which all book readers can use in the future as some kind of installable module.

[Thomas Wolmer]

I'll jump in and say that I agree with what jsager has been leaning towards - why is an additional format and an XML parser/"core engine" needed when the XML already is the machine-readable language and XML parsers (and more) come as a libraries available for "all" programming languages? (If a language doesn't have DOM, SAX and/or StAX parsers or some less standardized equivalents available, and preferrably something like XPath as well, it's the wrong language for this purpose.)

If you really need an XML -> CSV transformation, use XSLT. It should be quite easy since the existing Project Aon ones are available as "inspiration". That is, if the information can really be represented in a CSV file; I have my doubts...

One thing that a transformation of the navigation info into something more "accessible" than the XML files would be useful for is markup reviewing/proofing. If the choices in each section were summarized as simple text, then also people who are overwhelmed by the XML files could assist in checking that the markup is correct.

An item database (in XML of course) should definitely be part of the CNF project, and the same goes for the Action Chart since all reader software needs to deal with the same information. Remember that there's not just the classic Action Chart info to keep track of, like the stats and the items, it's also the "Have you ever visited ..." information. Those checkpoints need to be coded into the main XML files and recorded in the Action Chart.

Edit: Fixing typos and stuff

[alderaine]

Put simply, because those of us writing the book readers have no idea how to parse the XML to get the information we need!

There are a number of options (and there may be more)

1/ Publish guidelines on how to read the AON XML - some examples of code (any language would do) - a good example would be extracting information about a particular fight from a particular page.
This would be my preferred option as long as it gave all the information needed to novices (so far, none of the programmers of book readers have admited to using XML before)

2/ Publish a tool or transform/XSLT to help convert to formats used by the readers (Seriously, my book reader quite happily runs entirely from CSV files - they should still be in my staff area under 01fftd.csv for example)
Those of us who would use this format do not have the knowledge to write the transform!

The main reason I'm asking is that all applications I tried to open the AON XML in reported errors (some better than others at the job)

As to a language being wrong for the purpose, remember that the readers are being coded for every platform imagineable, many of which may not have any native XML support.

As with many cross-era projects, this one is encountering the slight barrier between the "modern" and the traditional. I can see both points of view and the needs of both sides.
It is absolutely essential (to my mind) that all the information ends up in the XML, but we can not really put "XML expert" as a prerequisite for writing book readers, so we should publish something to help people out. A set of guidelines & examples would be best.

[jsager]

I will happily provide sample code once I get the format worked out (and this is my main project, not the 7th sense website which should be a quick & dirty process).

[jsager]

Man, I got sucked up by work all day only to have our client push us off a week after I spent half my vacation working.

Anyway, I'll start on this tonight or tomorrow and should have something very shortly.

[alderaine]

That sounds perfectly normal for the software world.
Your help is very much appreciated - I'm using a text-based reader for the XML file at the moment (don't ask - it is rather bulky!) but it's written in such a way that I can just bolt your XML reader in instead, so that will be a perfect place to test it.
Mine's written in VB6, but I can convert most languages, or just call a DLL.

[alderaine]

Looking at the ITEM XML in FW in more detail, there are hints of some implementation issues.
The XML currently tries to highlight the items in the text itself. The problem is that this causes problems with pluralisms (i.e. 2 knives vs 1 knife)
We could continue highlighting items in the text, or we could keep the text pure and instead do all functional XML after the text. I would tend to recommend the latter approach.
If for some reason we do decide embedding XML in the text of the page is the right approach (can't think of a reason at the moment, but you never know) then it will need to be much more detailed, with the original text bracketed, but the actual identifiers for the item number, item name & quantity following after.
As a general question, do we think that items such as money, rounds of ammunition, meals, etc should be listed as items, or treated separately in the XML as a different type of thing (i.e. treated like an endurance modification since they are modifying quantities of a fixed thing on the action chart) - this latter approach would seem to work best with the idea of an XML action chart.
Let me know your thoughts

[alderaine]

jsager - one thought crosses my mind...
We are having problems trying to read the CURRENT XML format - if you are able to put together instructions for how to extract the information that is currently there (I still think battle is the best, as the most information is available) it would be extremely helpful.
I do not know how much we can change the core XML format. We can certainly make additions of new tag types to it, but I do not know if we can change the core layout - let me know if you feel this will be necessary and I'll talk to Jon first.
Assuming we do not change the core format, any guidelines you write for reading the current XML will remain valid and simply need the addition of the new tags as we implement them.

I have managed to read the existing XML via text manipulation, which is OK but not great in the long run due to limitations. Whenever I try to open it directly from the web using any XML manipulation tool, I get errors that will not allow my program to continue parsing it.

Many thanks

[jsager]

OK -

Is it possible for me to get staff space so that I can upload files? I have some work to present , but in order to present it properly for review I need to upload files.

Also, I believe that the common navigation project is going to grow to a size that it might make sense to give it its own forum category under the Playing Aid Software category... that will allow us to talk about the many branches of this project without having to try to clump it under one thread.

[Thomas Wolmer]

Jul 6, 2008 6:14:16 GMT jsager said:

Also, I believe that the common navigation project is going to grow to a size that it might make sense to give it its own forum category under the Playing Aid Software category... that will allow us to talk about the many branches of this project without having to try to clump it under one thread.

Wish granted!

I'd like someone to write a sticky post which explains in detail what the CNF is, so that they don't have to read through this whole thread to understand. Actually, it could be a sticky thread that also documents all decisions taken. (I could create this post myself, although not at this very moment.)

[jsager]

Thanks Thomas!

What sort of process do I need to go through in order to get space to store files so I can link to them?

Also, I'd be willing to write the CNF sticky post but only if alderaine wants me to, I don't want to steal his thunder.

[alderaine]

I've requested a staff area for you.

Please do go ahead and create the sticky - it will ensure we have a common understanding of the needs of the project

So far, the only concrete decision is that we will eventually update the AON XML with navigation information. As to how we get there... well, if you're happy to run with the project, I am more than happy for you to take it on