Friday, May 1, 2015

DTD (Document Type Definition) Validations During Message Transformations

While testing cXML Invoice service operations in a FSCM 9.2 application, I found some interesting information regarding DTD (Document Type Definition) validation of messages particularly when there are transformations.

CXML_INVOICE is a delivered service that allows third party systems to send invoice data using cXML protocol.



When I was testing the CXML_INVOICE.v1 (synchronous service operation) in a FSCM 9.2 environment (PeopleTools 8.54.07), the service operation was returning an error although there was no issues with XML message. If the same xml message was used to invoke the CXML_INVOICE.v1 service operation in a FSCM 9.1 (PeopleTools 8.50.10), the invoice was processed without any errors.

I enabled tracing at the integration gateway and also at the message transformation level to further troubleshoot this issue. This service operation CXML_INVOICE.v1 has a transformation on the inbound request:



I found that the xml message was getting wrapped in a CDATA section as follows:

Original Request:
<?xml version="1.0"?>
<!DOCTYPE cXML SYSTEM "http://xml.cXML.org/schemas/cXML/1.2.011/InvoiceDetail.dtd">
<cXML payloadID="TEST_INVOICE_123" timestamp="2015-03-27T06:19:25-05:00" xml:lang="en-US">
</cXML>

Request with CDATA wrapper:

<?xml version="1.0"?>
<data>
 <![CDATA[
<?xml version="1.0"?>
<!DOCTYPE cXML SYSTEM "http://xml.cXML.org/schemas/cXML/1.2.011/InvoiceDetail.dtd">
<cXML payloadID="TEST_INVOICE_123" timestamp="2015-03-27T06:19:25-05:00" xml:lang="en-US">
</cXML>
]]>
</data>


Due to this mysterious CDATA wrapping, the message transformation failed because the CDATA section changed the entire structure of the xml causing the xsl to fail while transforming the data to the appropriate fields in the destination format.

After some research on My Oracle Support, it was interesting to find the reason for the CDATA wrapping behavior. It turns out that the CDATA wrapper was caused by DTD validation issues on the Application Server.

Notice, <!DOCTYPE cXML SYSTEM "http://xml.cXML.org/schemas/cXML/1.2.011/InvoiceDetail.dtd"> in the original xml? This represents the Document Type Definition for the XML document. The application server would validate any inbound XML against the DTD if the DOCTYPE information is provided. The issue here is that the application server is unable to reach the location (possibly due to firewall issues) of the DTD for validation and therefore wrapped the inbound XML in a CDATA section.

On My Oracle Support there are a couple of documents which detail similar (not same) issues and provide some workaround options.

Refer:
E-IB: Asynchronous Inbound Message With DTD Reference Was Wrapped As PsNonXML, Causing a Transformation Failure (Doc ID 1941030.1)
E-IB: DTD Validation Causes Time-Outs or Significant Delays During XML Processing (Doc ID 850271.1)

I used the workaround option to suppress DTD validation at the application server level using an Integration Broker property on the configuration file (psappsrv.cfg):

[Integration Broker]
;=========================================================================
; General settings for the Integration Broker
;=========================================================================
DTD Lookup=0

Once I made this change, the DTD validation was bypassed and subsequently the CDATA wrapper was also suppressed resulting in the transformation app engine to work as expected.

Note: This DTD Lookup IB configuration on the app server configuration file should not be confused with the ig.dtdLookup property on the IB gateway properties file.

I thought this was worth a blog post because I could not find much information on the DTD Lookup Integration Broker application server configuration in PeopleBooks (other than on My Oracle Support). Hope you find this useful!

The other workarounds that were suggested were:
  1. Remove the DTD URL from the XML documents. I don't think this is a viable option particularly in integrations that deal with third party systems where we cannot dictate the syntax and semantics.
  2. Place the DTD document that cannot be reached at its native location onto the same path on another HTTP server that is accessible to the app server (could be even the PIA web server). I found this to be a very interesting solution if we run into this issue and suppressing DTD validations is not an option. Please refer Doc ID 850271.1 for more details.

2 comments:

  1. It's very interesting, thanks for sharing.
    CXML Punchout

    ReplyDelete
  2. At COEPD we are glad to announce Web Designing Internship Programs (Self sponsored) for professionals who want to have hands on experience. Associations with IT Companies, at present we are providing this program in COEPD Hyderabad premises. We anticipate finest web designing technology covering Photoshop, HTML, CSS, HTML5, CSS3, JavaScript, Boot Strap, and JQuery. We tutor participants to be solution providers and creative engineers. This internship is intelligently dedicated to our enthusiastic and passionate participants mostly acknowledging and appreciating the fact that they are on the path of making a career in Web Designing. This internship is designed to fortify that in addition to gaining the vital theoretical knowledge, the readers gain sufficient hands-on practice and practical know-how to master the essence of Web Designing profession. More than a training institute, COEPD today stands differentiated as a mission to help you "Build your dream career" - COEPD way.

    http://www.coepd.com/WebDesigning-Internships.html

    ReplyDelete