Web Host Xml Data Pump

Web Host now accepts real time XML data updates. This enables you to load, real time - into Collect!, the latest account status, notes, contacts, financials and other information provided by any external system able to send XML data.

When Web Host is started in a special mode /W3 it expects XML data for incoming data packets. The first XML tag in the incoming data packet tells Collect! the name of an import map to run when loading the data into the Collect! database.

Web Host converts the incoming XML data into a standard comma-separated CSV file. The order of the tags in the XML data packet determines the order of the columns in the CSV file. So the import map in Collect! must be a comma-separated import map with the same layout as the fields in the XML packet.

The intermediate CSV file that Collect! creates is placed in Collect!'s import folder. It is removed after it has been read successfully. If Web Host is started with the /whl command, the CSV files are not removed. This is very useful for troubleshooting.

How This Works

Collect! simply converts the XML data into a CSV file based on the XML tags. Then the import functionality in Collect! simply reads this CSV file and imports it based on the design of the import map. So the import map must match the layout of the XML data so that all the columns line up correctly when the CSV file is created and when it is read by Collect!'s import functions.

The Web Host shortcut properties are modified to allow direct access to the other system. This requires changing the /w1 to a /w3. This means that a login is not required and direct submits from the other system are allowed.

The external system submits the file, based on the form, to the IP address of the Web Host server.

If the client is actually using the Web Host, you would need to set up shortcuts to use different ports, one with the /w1 and one with the /w3.

1. After the HTTP header, we check for the first XML tag, for example text <noblexml>. If that is found then we set a flag isXML and invoke isXML parsing code.

2. The <noblexml>, for example, becomes the 'importmap' wrapper to select the noblexml import. Thus for this to work we need an import in Collect! named 'nobleXML'.

3. The XML tags are parsed to load each tag element into a CSV column. The name of the tag is not important but the field order is very important. It is the order of the fields that is used to create the CSV columns and these have to match the import map (in this case, 'nobleXML'.)

The elements 'redirectto' and 'reportname' will be used if provided as in standard Web Host behavior.

redirectto Requests a specific page.

reportname Requests a report to be run after the import.

HTML Request Example

Submit XML File Please ...

<form name="filexml"
action="import"
enctype="multipart/form-data"
method="post">
<input type="file" name="myFile.xml">
<input type="submit" value="Upload File">
</form>

Example Xml Data In Myfile.Xml

<noblexml>
<listid>101</listid>
<appl>SALE</appl>
<clientaccount>1400</clientaccount>
<filler3>23434</filler3>
<call_date>10/12/2005</call_date>
<call_time>143245</call_time>
<country_id>1</country_id>
<areacode>404</areacode>
< phone>8511221</phone>
<tsr>TAC</tsr>
<status>LD</status>
<addi_status>AA</addi_status>
</noblexml>

Xml Data Pump Technical Notes

1. XML tags are case sensitive. </Addi_status> does NOT close <addi_status>.

2. The parser terminates on non closed fields.

3. You must have an import map named 'noblexml', for example, to import the data, i.e. an import map whose name is the same as the first tag found in the incoming file.

4. Web Host must be started with the /W3 switch to bypass authentication when using Web Host. This change makes WebUserOK() return a 1 which goes into DoHttpRequest() looking for the POST or GET command.

5. If needed, you may set a maximum packet size in Database Preferences in the Web Host Session Mem KB field.

6. Collect! does not parse packets sent as unicode or UTF-8. They must be sent in an ASCII character set, such as ISO8859-1 or one of the other sets, such as ISO8859-13, which works for Baltic countries.

Sample Packet

POST /import HTTP/1.1\r\n
Accept:\r\n
Content-Length: PAYLOAD_SIZE\r\n\r\n
PAYLOAD_DATA

The data packet must be text/xml, not application/xml.

PAYLOAD_DATA does not contain a NULL terminator and is exactly PAYLOAD_SIZE bytes in length. Collect! XML handling code will be invoked by the tags <noblexml> .... </noblexml>.

Operator Setup

Set the Web Host up with an operator so that when you log in as that operator your Web Host can start up in Web Host mode.

Operator - Type must be "Web Host" to enable it.

Operator - Security must be "Web Host XML data pump." This will start Web Host with the /W3 switch enabled.

Operator - Detail - Just type the port number in the field.

Note 1 = /s80 - Not filling this in defaults to port 80.

Note 2 = /y443 - Not filling this in defaults to port 443.

Troubleshooting With Web Host Packet Logging

This shortcut argument, /whl, is used for troubleshooting transmission of Web Host data. The /whl command will also prevent the deletion of the temporary CSV files that Collect! uses for translating the XML data into CSV format for importing into Collect!.

Example:
C:\CV11\CV11.exe /S80 /W3 /y443 /whl
Web Host Packet Logging Enabled

This will generate a text file in your CV11\bin folder, i.e. "webhostpacketlog.txt." This is for advanced developer troubleshooting. This file can be used to reproduce HTTP headers and payload content, for instance in XML data pump troubleshooting.

This will also allow you to log SSL encrypted sessions too, since Collect! writes to these files after the decryption phase.

Internationalization

Collect! does not parse packets sent as unicode or UTF-8. They must be sent in an ASCII character set, such as ISO8859-1 or one of the other sets, such as ISO8859-13, which works for Baltic countries.

See Also

- Web Host Topics