smooks (EDI) + talend

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

smooks (EDI) + talend

rmg78
Hi there,
I'm trying to use smooks in a talend flow... with some serious problems and I need some help:

1) The smooks component for talend is based on an older version of smooks (1.2.4). Looking on the project site... it appears to be dead. Has anybody tried to upgrade the component al least at version 1.4?

2) The EDI files i'm trying to read have several types of messages. I've read that it's possible to manage this but in version 1.4 (interchange). Two questions about this: a)can i achieve this in version 1.2.4? b)i do not need to parse all the messages, just on or two types. For example... I've the following type of messages:

{file begin}
                UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
                UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
                UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
                UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
{file end}

I only want to process the CCPRRR message, who has inside... hundreds of items. I don't want to generate the mappings for the other types. Is it possible?

3) In the above example, i've converted the delimiters. In the real file, the delimiters are hexadecimal characters. Example: instead of plus (+), hay have 1D (hex). Couldn't find to tell smooks config file to use this kind of separators. Any ideas? i've files with more than 1Gb, so replace is not an option.

Enough for the moment, any help would be apreciated. The type of messages i'm using are from the air industry (amadeus - altea).
thanks
max.-
Reply | Threaded
Open this post in threaded view
|

Re: smooks (EDI) + talend

Renat Zubairov-4
Hello Max,

Concerning Talend-Smooks integration dedicated to EDI parsing I can give
you some sneak preview of it. Here are some screenshots I made with
upcoming Talend Open Studio 4.2.0:

http://screencast.com/t/4mDhoHZw7
http://screencast.com/t/rjijRpjB
http://screencast.com/t/2Wlfp10ye


UN/EDIFACT interchange support was introduced in Smooks 1.4, I don't think
you could backport it easily to 1.2.4, however latest Talend components
(tEDItoXML) are based on Smooks 1.4 therefore fully support UN/EDIFACT
interchanges.

Once your message is transformed to XML you could select any part that you
are interested in.

Concerning the specific separator char, I think it's doable, but AFAK
separators are specified in UNH headers, I suppose they should be also
treated well by 1.4 EDI Parser, but I might be wrong here. Anyway you can
programmatically override the default separators.

Best regards,
Renat Zubairov

--
Product Owner ESB Tooling
Talend Application Integration Division





Am 28.04.11 22:14 schrieb "rmg78" unter <[hidden email]>:

>
>Hi there,
>I'm trying to use smooks in a talend flow... with some serious problems
>and
>I need some help:
>
>1) The smooks component for talend is based on an older version of smooks
>(1.2.4). Looking on the project site... it appears to be dead. Has anybody
>tried to upgrade the component al least at version 1.4?
>
>2) The EDI files i'm trying to read have several types of messages. I've
>read that it's possible to manage this but in version 1.4 (interchange).
>Two
>questions about this: a)can i achieve this in version 1.2.4? b)i do not
>need
>to parse all the messages, just on or two types. For example... I've the
>following type of messages:
>
>{file begin}
>        UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
>        UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
>        UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
>        UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
>{file end}
>
>I only want to process the CCPRRR message, who has inside... hundreds of
>items. I don't want to generate the mappings for the other types. Is it
>possible?
>
>3) In the above example, i've converted the delimiters. In the real file,
>the delimiters are hexadecimal characters. Example: instead of plus (+),
>hay
>have 1D (hex). Couldn't find to tell smooks config file to use this kind
>of
>separators. Any ideas? i've files with more than 1Gb, so replace is not an
>option.
>
>Enough for the moment, any help would be apreciated. The type of messages
>i'm using are from the air industry (amadeus - altea).
>thanks
>max.-
>--
>View this message in context:
>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31500077.html
>Sent from the milyn - user mailing list archive at Nabble.com.
>
>
>---------------------------------------------------------------------
>To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: smooks (EDI) + talend

rmg78
Hi Renat

Like I said before, i wanted to test it before give you an answer, and I did it: awesome.
I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with the amadeus mappings and after a few little problems it worked like a charm.
I had to touch the mapping a little... for the upgrade from smooks 1.2.4 to 1.4 (nodeTypeRef, remove of the UNH mapping, ...) and nothing more.

Going back to the original questions, 1 & 2 are solved (smooks discards the mappings not included in the jar, excelent), and I'm working on number 3.

By the way, in the screenshots you uploaded you have the EDIFACT bullet in the metadata and the mapping editor -> the RC3 dons't include this, need to wait to 4.2.0 right?

For future questions about talend flow originated from this component, do I need to write them in another forum? can you tell me which one.

Thanks again!
max.-

From

Renat Zubairov-4 wrote
Hello Max,

Concerning Talend-Smooks integration dedicated to EDI parsing I can give
you some sneak preview of it. Here are some screenshots I made with
upcoming Talend Open Studio 4.2.0:

http://screencast.com/t/4mDhoHZw7
http://screencast.com/t/rjijRpjB
http://screencast.com/t/2Wlfp10ye


UN/EDIFACT interchange support was introduced in Smooks 1.4, I don't think
you could backport it easily to 1.2.4, however latest Talend components
(tEDItoXML) are based on Smooks 1.4 therefore fully support UN/EDIFACT
interchanges.

Once your message is transformed to XML you could select any part that you
are interested in.

Concerning the specific separator char, I think it's doable, but AFAK
separators are specified in UNH headers, I suppose they should be also
treated well by 1.4 EDI Parser, but I might be wrong here. Anyway you can
programmatically override the default separators.

Best regards,
Renat Zubairov

--
Product Owner ESB Tooling
Talend Application Integration Division





Am 28.04.11 22:14 schrieb "rmg78" unter <rmgiorgi@gmail.com>:

>
>Hi there,
>I'm trying to use smooks in a talend flow... with some serious problems
>and
>I need some help:
>
>1) The smooks component for talend is based on an older version of smooks
>(1.2.4). Looking on the project site... it appears to be dead. Has anybody
>tried to upgrade the component al least at version 1.4?
>
>2) The EDI files i'm trying to read have several types of messages. I've
>read that it's possible to manage this but in version 1.4 (interchange).
>Two
>questions about this: a)can i achieve this in version 1.2.4? b)i do not
>need
>to parse all the messages, just on or two types. For example... I've the
>following type of messages:
>
>{file begin}
>        UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
>        UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
>        UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
>        UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
>{file end}
>
>I only want to process the CCPRRR message, who has inside... hundreds of
>items. I don't want to generate the mappings for the other types. Is it
>possible?
>
>3) In the above example, i've converted the delimiters. In the real file,
>the delimiters are hexadecimal characters. Example: instead of plus (+),
>hay
>have 1D (hex). Couldn't find to tell smooks config file to use this kind
>of
>separators. Any ideas? i've files with more than 1Gb, so replace is not an
>option.
>
>Enough for the moment, any help would be apreciated. The type of messages
>i'm using are from the air industry (amadeus - altea).
>thanks
>max.-
>--
>View this message in context:
>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31500077.html
>Sent from the milyn - user mailing list archive at Nabble.com.
>
>
>---------------------------------------------------------------------
>To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: smooks (EDI) + talend

rmg78
Oops!!! I made a mistake on the previous test, smooks doesn't discard the unmapped messages.
Is there a solution for my question number two ? i mean... having:

{file begin}
        UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
        UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
        UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
        UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
{file end}

just parse the CCPRRR discarding the other messages and having no mapping of them in the jar?

thanks
max.-

rmg78 wrote
Hi Renat

Like I said before, i wanted to test it before give you an answer, and I did it: awesome.
I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with the amadeus mappings and after a few little problems it worked like a charm.
I had to touch the mapping a little... for the upgrade from smooks 1.2.4 to 1.4 (nodeTypeRef, remove of the UNH mapping, ...) and nothing more.

Going back to the original questions, 1 & 2 are solved (smooks discards the mappings not included in the jar, excelent), and I'm working on number 3.

By the way, in the screenshots you uploaded you have the EDIFACT bullet in the metadata and the mapping editor -> the RC3 dons't include this, need to wait to 4.2.0 right?

For future questions about talend flow originated from this component, do I need to write them in another forum? can you tell me which one.

Thanks again!
max.-

From

Renat Zubairov-4 wrote
Hello Max,

Concerning Talend-Smooks integration dedicated to EDI parsing I can give
you some sneak preview of it. Here are some screenshots I made with
upcoming Talend Open Studio 4.2.0:

http://screencast.com/t/4mDhoHZw7
http://screencast.com/t/rjijRpjB
http://screencast.com/t/2Wlfp10ye


UN/EDIFACT interchange support was introduced in Smooks 1.4, I don't think
you could backport it easily to 1.2.4, however latest Talend components
(tEDItoXML) are based on Smooks 1.4 therefore fully support UN/EDIFACT
interchanges.

Once your message is transformed to XML you could select any part that you
are interested in.

Concerning the specific separator char, I think it's doable, but AFAK
separators are specified in UNH headers, I suppose they should be also
treated well by 1.4 EDI Parser, but I might be wrong here. Anyway you can
programmatically override the default separators.

Best regards,
Renat Zubairov

--
Product Owner ESB Tooling
Talend Application Integration Division





Am 28.04.11 22:14 schrieb "rmg78" unter <rmgiorgi@gmail.com>:

>
>Hi there,
>I'm trying to use smooks in a talend flow... with some serious problems
>and
>I need some help:
>
>1) The smooks component for talend is based on an older version of smooks
>(1.2.4). Looking on the project site... it appears to be dead. Has anybody
>tried to upgrade the component al least at version 1.4?
>
>2) The EDI files i'm trying to read have several types of messages. I've
>read that it's possible to manage this but in version 1.4 (interchange).
>Two
>questions about this: a)can i achieve this in version 1.2.4? b)i do not
>need
>to parse all the messages, just on or two types. For example... I've the
>following type of messages:
>
>{file begin}
>        UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
>        UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
>        UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
>        UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
>{file end}
>
>I only want to process the CCPRRR message, who has inside... hundreds of
>items. I don't want to generate the mappings for the other types. Is it
>possible?
>
>3) In the above example, i've converted the delimiters. In the real file,
>the delimiters are hexadecimal characters. Example: instead of plus (+),
>hay
>have 1D (hex). Couldn't find to tell smooks config file to use this kind
>of
>separators. Any ideas? i've files with more than 1Gb, so replace is not an
>option.
>
>Enough for the moment, any help would be apreciated. The type of messages
>i'm using are from the air industry (amadeus - altea).
>thanks
>max.-
>--
>View this message in context:
>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31500077.html
>Sent from the milyn - user mailing list archive at Nabble.com.
>
>
>---------------------------------------------------------------------
>To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: smooks (EDI) + talend

Renat Zubairov-4
In reply to this post by rmg78
Hello Max,

See my comments below:

> I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with
>the
> amadeus mappings and after a few little problems it worked like a charm.


If I understood you correctly you deployed additional UN/EDIFACT mapping
JAR file to the component, how did you do it? Do you think we should
extend component so that such use-case would be supported.

> By the way, in the screenshots you uploaded you have the EDIFACT bullet
>in
> the metadata and the mapping editor -> the RC3 dons't include this, need
>to
> wait to 4.2.0 right?

These screenshots were made not from the tEDIFACTtoXML component but from
the tEDI component which is a part of commercial Talend offering. In
commercial edition (TIS) you would be able to define a project-wide common
metadata based on UN/EDIFACT mappings which would directly map to flat
Talend schemas (without any XML in-between). Generally same Smooks
functionality, just a little more convenient.

> For future questions about talend flow originated from this component,
>do I
> need to write them in another forum? can you tell me which one.

Sure, you can ask your question on Forum: http://talendforge.org/forum/

Best regards,
Renat Zubairov

--
Product Owner ESB Tooling
Talend Application Integration Division







Am 29.04.11 17:08 schrieb "rmg78" unter <[hidden email]>:

>
>Hi Renat
>
>Like I said before, i wanted to test it before give you an answer, and I
>did
>it: awesome.
>I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with
>the
>amadeus mappings and after a few little problems it worked like a charm.
>I had to touch the mapping a little... for the upgrade from smooks 1.2.4
>to
>1.4 (nodeTypeRef, remove of the UNH mapping, ...) and nothing more.
>
>Going back to the original questions, 1 & 2 are solved (smooks discards
>the
>mappings not included in the jar, excelent), and I'm working on number 3.
>
>By the way, in the screenshots you uploaded you have the EDIFACT bullet in
>the metadata and the mapping editor -> the RC3 dons't include this, need
>to
>wait to 4.2.0 right?
>
>For future questions about talend flow originated from this component, do
>I
>need to write them in another forum? can you tell me which one.
>
>Thanks again!
>max.-
>
>From
>
>
>Renat Zubairov-4 wrote:
>>
>> Hello Max,
>>
>> Concerning Talend-Smooks integration dedicated to EDI parsing I can give
>> you some sneak preview of it. Here are some screenshots I made with
>> upcoming Talend Open Studio 4.2.0:
>>
>> http://screencast.com/t/4mDhoHZw7
>> http://screencast.com/t/rjijRpjB
>> http://screencast.com/t/2Wlfp10ye
>>
>>
>> UN/EDIFACT interchange support was introduced in Smooks 1.4, I don't
>>think
>> you could backport it easily to 1.2.4, however latest Talend components
>> (tEDItoXML) are based on Smooks 1.4 therefore fully support UN/EDIFACT
>> interchanges.
>>
>> Once your message is transformed to XML you could select any part that
>>you
>> are interested in.
>>
>> Concerning the specific separator char, I think it's doable, but AFAK
>> separators are specified in UNH headers, I suppose they should be also
>> treated well by 1.4 EDI Parser, but I might be wrong here. Anyway you
>>can
>> programmatically override the default separators.
>>
>> Best regards,
>> Renat Zubairov
>>
>> --
>> Product Owner ESB Tooling
>> Talend Application Integration Division
>>
>>
>>
>>
>>
>> Am 28.04.11 22:14 schrieb "rmg78" unter <[hidden email]>:
>>
>>>
>>>Hi there,
>>>I'm trying to use smooks in a talend flow... with some serious problems
>>>and
>>>I need some help:
>>>
>>>1) The smooks component for talend is based on an older version of
>>>smooks
>>>(1.2.4). Looking on the project site... it appears to be dead. Has
>>>anybody
>>>tried to upgrade the component al least at version 1.4?
>>>
>>>2) The EDI files i'm trying to read have several types of messages. I've
>>>read that it's possible to manage this but in version 1.4 (interchange).
>>>Two
>>>questions about this: a)can i achieve this in version 1.2.4? b)i do not
>>>need
>>>to parse all the messages, just on or two types. For example... I've the
>>>following type of messages:
>>>
>>>{file begin}
>>>        UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
>>>        UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
>>>        UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
>>>        UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
>>>{file end}
>>>
>>>I only want to process the CCPRRR message, who has inside... hundreds of
>>>items. I don't want to generate the mappings for the other types. Is it
>>>possible?
>>>
>>>3) In the above example, i've converted the delimiters. In the real
>>>file,
>>>the delimiters are hexadecimal characters. Example: instead of plus (+),
>>>hay
>>>have 1D (hex). Couldn't find to tell smooks config file to use this kind
>>>of
>>>separators. Any ideas? i've files with more than 1Gb, so replace is not
>>>an
>>>option.
>>>
>>>Enough for the moment, any help would be apreciated. The type of
>>>messages
>>>i'm using are from the air industry (amadeus - altea).
>>>thanks
>>>max.-
>>>--
>>>View this message in context:
>>>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31500077.ht
>>>ml
>>>Sent from the milyn - user mailing list archive at Nabble.com.
>>>
>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe from this list, please visit:
>>>
>>>    http://xircles.codehaus.org/manage_email
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>>
>>
>
>--
>View this message in context:
>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31506305.html
>Sent from the milyn - user mailing list archive at Nabble.com.
>
>
>---------------------------------------------------------------------
>To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: smooks (EDI) + talend

Renat Zubairov-4
In reply to this post by rmg78
Hello Max,

The problem you are running in is a Smooks UN/EDIFACT parser limitation.
Right now parser expects one _interchange_ (UNH...UNZ) in one file. So far
it was always the case.
Is this limitation a significant issue for you? May be if you could
describe your use-case it will be more clear for us why and when we could
run into that limitation.

Best regards,
Renat Zubairov

--
Product Owner ESB Tooling
Talend Application Integration Division





Am 29.04.11 20:41 schrieb "rmg78" unter <[hidden email]>:

>
>Oops!!! I made a mistake on the previous test, smooks doesn't discard the
>unmapped messages.
>Is there a solution for my question number two ? i mean... having:
>
>{file begin}
>        UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
>        UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
>        UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
>        UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
>{file end}
>
>just parse the CCPRRR discarding the other messages and having no mapping
>of
>them in the jar?
>
>thanks
>max.-
>
>
>rmg78 wrote:
>>
>> Hi Renat
>>
>> Like I said before, i wanted to test it before give you an answer, and I
>> did it: awesome.
>> I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with
>> the amadeus mappings and after a few little problems it worked like a
>> charm.
>> I had to touch the mapping a little... for the upgrade from smooks 1.2.4
>> to 1.4 (nodeTypeRef, remove of the UNH mapping, ...) and nothing more.
>>
>> Going back to the original questions, 1 & 2 are solved (smooks discards
>> the mappings not included in the jar, excelent), and I'm working on
>>number
>> 3.
>>
>> By the way, in the screenshots you uploaded you have the EDIFACT bullet
>>in
>> the metadata and the mapping editor -> the RC3 dons't include this, need
>> to wait to 4.2.0 right?
>>
>> For future questions about talend flow originated from this component,
>>do
>> I need to write them in another forum? can you tell me which one.
>>
>> Thanks again!
>> max.-
>>
>> From
>>
>>
>> Renat Zubairov-4 wrote:
>>>
>>> Hello Max,
>>>
>>> Concerning Talend-Smooks integration dedicated to EDI parsing I can
>>>give
>>> you some sneak preview of it. Here are some screenshots I made with
>>> upcoming Talend Open Studio 4.2.0:
>>>
>>> http://screencast.com/t/4mDhoHZw7
>>> http://screencast.com/t/rjijRpjB
>>> http://screencast.com/t/2Wlfp10ye
>>>
>>>
>>> UN/EDIFACT interchange support was introduced in Smooks 1.4, I don't
>>> think
>>> you could backport it easily to 1.2.4, however latest Talend components
>>> (tEDItoXML) are based on Smooks 1.4 therefore fully support UN/EDIFACT
>>> interchanges.
>>>
>>> Once your message is transformed to XML you could select any part that
>>> you
>>> are interested in.
>>>
>>> Concerning the specific separator char, I think it's doable, but AFAK
>>> separators are specified in UNH headers, I suppose they should be also
>>> treated well by 1.4 EDI Parser, but I might be wrong here. Anyway you
>>>can
>>> programmatically override the default separators.
>>>
>>> Best regards,
>>> Renat Zubairov
>>>
>>> --
>>> Product Owner ESB Tooling
>>> Talend Application Integration Division
>>>
>>>
>>>
>>>
>>>
>>> Am 28.04.11 22:14 schrieb "rmg78" unter <[hidden email]>:
>>>
>>>>
>>>>Hi there,
>>>>I'm trying to use smooks in a talend flow... with some serious problems
>>>>and
>>>>I need some help:
>>>>
>>>>1) The smooks component for talend is based on an older version of
>>>>smooks
>>>>(1.2.4). Looking on the project site... it appears to be dead. Has
>anybody
>>>>tried to upgrade the component al least at version 1.4?
>>>>
>>>>2) The EDI files i'm trying to read have several types of messages.
>>>>I've
>>>>read that it's possible to manage this but in version 1.4
>>>>(interchange).
>>>>Two
>>>>questions about this: a)can i achieve this in version 1.2.4? b)i do not
>>>>need
>>>>to parse all the messages, just on or two types. For example... I've
>>>>the
>>>>following type of messages:
>>>>
>>>>{file begin}
>>>>        UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
>>>>        UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
>>>>        UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
>>>>        UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
>>>>{file end}
>>>>
>>>>I only want to process the CCPRRR message, who has inside... hundreds
>>>>of
>>>>items. I don't want to generate the mappings for the other types. Is it
>>>>possible?
>>>>
>>>>3) In the above example, i've converted the delimiters. In the real
>>>>file,
>>>>the delimiters are hexadecimal characters. Example: instead of plus
>>>>(+),
>>>>hay
>>>>have 1D (hex). Couldn't find to tell smooks config file to use this
>>>>kind
>>>>of
>>>>separators. Any ideas? i've files with more than 1Gb, so replace is not
>an
>>>>option.
>>>>
>>>>Enough for the moment, any help would be apreciated. The type of
>>>>messages
>>>>i'm using are from the air industry (amadeus - altea).
>>>>thanks
>>>>max.-
>>>>--
>>>>View this message in context:
>>>>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31500077.h
>>>>tml
>>>>Sent from the milyn - user mailing list archive at Nabble.com.
>>>>
>>>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe from this list, please visit:
>>>>
>>>>    http://xircles.codehaus.org/manage_email
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>     http://xircles.codehaus.org/manage_email
>>>
>>>
>>>
>>>
>>
>>
>
>--
>View this message in context:
>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31507923.html
>Sent from the milyn - user mailing list archive at Nabble.com.
>
>
>---------------------------------------------------------------------
>To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: smooks (EDI) + talend

rmg78
In reply to this post by Renat Zubairov-4
Hi Renat,

In order to deploy the additional mappings i looked one of the existing jars, and based on that i replicated the content with my mappings.
After that, i copied the jar in the talend component and in the lib/java folder and edited the component's files (*.properties and *.xml). The other thing i changed, was the configuration/ComponentsCache.javacache file, appending the new library.
With these changes, I can see my "amadeus" package in the component's combo.

I do not consider that these mappings should be integrated in TOS because they aren't standard mappings, the mappings are property of amadeus.  

In regard of TIS, i'll see it with my project manager.
Thanks a lot for your support,
max.-

Renat Zubairov-4 wrote
Hello Max,

See my comments below:

> I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with
>the
> amadeus mappings and after a few little problems it worked like a charm.


If I understood you correctly you deployed additional UN/EDIFACT mapping
JAR file to the component, how did you do it? Do you think we should
extend component so that such use-case would be supported.

> By the way, in the screenshots you uploaded you have the EDIFACT bullet
>in
> the metadata and the mapping editor -> the RC3 dons't include this, need
>to
> wait to 4.2.0 right?

These screenshots were made not from the tEDIFACTtoXML component but from
the tEDI component which is a part of commercial Talend offering. In
commercial edition (TIS) you would be able to define a project-wide common
metadata based on UN/EDIFACT mappings which would directly map to flat
Talend schemas (without any XML in-between). Generally same Smooks
functionality, just a little more convenient.

> For future questions about talend flow originated from this component,
>do I
> need to write them in another forum? can you tell me which one.

Sure, you can ask your question on Forum: http://talendforge.org/forum/

Best regards,
Renat Zubairov

--
Product Owner ESB Tooling
Talend Application Integration Division







Am 29.04.11 17:08 schrieb "rmg78" unter <rmgiorgi@gmail.com>:

>
>Hi Renat
>
>Like I said before, i wanted to test it before give you an answer, and I
>did
>it: awesome.
>I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with
>the
>amadeus mappings and after a few little problems it worked like a charm.
>I had to touch the mapping a little... for the upgrade from smooks 1.2.4
>to
>1.4 (nodeTypeRef, remove of the UNH mapping, ...) and nothing more.
>
>Going back to the original questions, 1 & 2 are solved (smooks discards
>the
>mappings not included in the jar, excelent), and I'm working on number 3.
>
>By the way, in the screenshots you uploaded you have the EDIFACT bullet in
>the metadata and the mapping editor -> the RC3 dons't include this, need
>to
>wait to 4.2.0 right?
>
>For future questions about talend flow originated from this component, do
>I
>need to write them in another forum? can you tell me which one.
>
>Thanks again!
>max.-
>
>From
>
>
>Renat Zubairov-4 wrote:
>>
>> Hello Max,
>>
>> Concerning Talend-Smooks integration dedicated to EDI parsing I can give
>> you some sneak preview of it. Here are some screenshots I made with
>> upcoming Talend Open Studio 4.2.0:
>>
>> http://screencast.com/t/4mDhoHZw7
>> http://screencast.com/t/rjijRpjB
>> http://screencast.com/t/2Wlfp10ye
>>
>>
>> UN/EDIFACT interchange support was introduced in Smooks 1.4, I don't
>>think
>> you could backport it easily to 1.2.4, however latest Talend components
>> (tEDItoXML) are based on Smooks 1.4 therefore fully support UN/EDIFACT
>> interchanges.
>>
>> Once your message is transformed to XML you could select any part that
>>you
>> are interested in.
>>
>> Concerning the specific separator char, I think it's doable, but AFAK
>> separators are specified in UNH headers, I suppose they should be also
>> treated well by 1.4 EDI Parser, but I might be wrong here. Anyway you
>>can
>> programmatically override the default separators.
>>
>> Best regards,
>> Renat Zubairov
>>
>> --
>> Product Owner ESB Tooling
>> Talend Application Integration Division
>>
>>
>>
>>
>>
>> Am 28.04.11 22:14 schrieb "rmg78" unter <rmgiorgi@gmail.com>:
>>
>>>
>>>Hi there,
>>>I'm trying to use smooks in a talend flow... with some serious problems
>>>and
>>>I need some help:
>>>
>>>1) The smooks component for talend is based on an older version of
>>>smooks
>>>(1.2.4). Looking on the project site... it appears to be dead. Has
>>>anybody
>>>tried to upgrade the component al least at version 1.4?
>>>
>>>2) The EDI files i'm trying to read have several types of messages. I've
>>>read that it's possible to manage this but in version 1.4 (interchange).
>>>Two
>>>questions about this: a)can i achieve this in version 1.2.4? b)i do not
>>>need
>>>to parse all the messages, just on or two types. For example... I've the
>>>following type of messages:
>>>
>>>{file begin}
>>>        UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
>>>        UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
>>>        UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
>>>        UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
>>>{file end}
>>>
>>>I only want to process the CCPRRR message, who has inside... hundreds of
>>>items. I don't want to generate the mappings for the other types. Is it
>>>possible?
>>>
>>>3) In the above example, i've converted the delimiters. In the real
>>>file,
>>>the delimiters are hexadecimal characters. Example: instead of plus (+),
>>>hay
>>>have 1D (hex). Couldn't find to tell smooks config file to use this kind
>>>of
>>>separators. Any ideas? i've files with more than 1Gb, so replace is not
>>>an
>>>option.
>>>
>>>Enough for the moment, any help would be apreciated. The type of
>>>messages
>>>i'm using are from the air industry (amadeus - altea).
>>>thanks
>>>max.-
>>>--
>>>View this message in context:
>>>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31500077.ht
>>>ml
>>>Sent from the milyn - user mailing list archive at Nabble.com.
>>>
>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe from this list, please visit:
>>>
>>>    http://xircles.codehaus.org/manage_email
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>>
>>
>
>--
>View this message in context:
>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31506305.html
>Sent from the milyn - user mailing list archive at Nabble.com.
>
>
>---------------------------------------------------------------------
>To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: smooks (EDI) + talend

rmg78
In reply to this post by Renat Zubairov-4
Renat,
The real life case is the following:

1- I receive a tar-zipped file, with several .data files. These files are sent from the amadeus server directly to our filesystem.
2- each .data file contains several edifact messages, being one of them the CCPRRR that i'm interested in. i need to discard the rest because i don't need them, and i don't have such mappings.

As i explained, each file contains the messages as:

Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1'
Line 2: UNH+1+BIDBRR:07:2:1A+999999'  ... UNT+761+1'
Line 3: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+762+1'
Line 4: UNH+1+MHISRR:05:0:1A+999999'  ... UNT+763+1'
Line 5: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+764+1'
...
With the component that i've now, if the file contains only CCPRRR messages it works fine, but in the above example... fails saying:

Starting job EDIToXML_CCPRRR at 15:26 02/05/2011.

[statistics] connecting to socket on port 3813
[statistics] connected
Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping model.
Exception in component tFileOutputXML_1
java.lang.NullPointerException
        at aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(EDIToXML_CCPRRR.java:649)
        at aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CCPRRR.java:911)
        at aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.java:779)
[statistics] disconnected
Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1]

What i don't want is to map every kind of message they sent in the files. any ideas?

Thanks
max

Renat Zubairov-4 wrote
Hello Max,

The problem you are running in is a Smooks UN/EDIFACT parser limitation.
Right now parser expects one _interchange_ (UNH...UNZ) in one file. So far
it was always the case.
Is this limitation a significant issue for you? May be if you could
describe your use-case it will be more clear for us why and when we could
run into that limitation.

Best regards,
Renat Zubairov

--
Product Owner ESB Tooling
Talend Application Integration Division





Am 29.04.11 20:41 schrieb "rmg78" unter <rmgiorgi@gmail.com>:

>
>Oops!!! I made a mistake on the previous test, smooks doesn't discard the
>unmapped messages.
>Is there a solution for my question number two ? i mean... having:
>
>{file begin}
>        UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
>        UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
>        UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
>        UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
>{file end}
>
>just parse the CCPRRR discarding the other messages and having no mapping
>of
>them in the jar?
>
>thanks
>max.-
>
>
>rmg78 wrote:
>>
>> Hi Renat
>>
>> Like I said before, i wanted to test it before give you an answer, and I
>> did it: awesome.
>> I've installed the RC3, used the tEDIFACTtoXML, defined my own jar with
>> the amadeus mappings and after a few little problems it worked like a
>> charm.
>> I had to touch the mapping a little... for the upgrade from smooks 1.2.4
>> to 1.4 (nodeTypeRef, remove of the UNH mapping, ...) and nothing more.
>>
>> Going back to the original questions, 1 & 2 are solved (smooks discards
>> the mappings not included in the jar, excelent), and I'm working on
>>number
>> 3.
>>
>> By the way, in the screenshots you uploaded you have the EDIFACT bullet
>>in
>> the metadata and the mapping editor -> the RC3 dons't include this, need
>> to wait to 4.2.0 right?
>>
>> For future questions about talend flow originated from this component,
>>do
>> I need to write them in another forum? can you tell me which one.
>>
>> Thanks again!
>> max.-
>>
>> From
>>
>>
>> Renat Zubairov-4 wrote:
>>>
>>> Hello Max,
>>>
>>> Concerning Talend-Smooks integration dedicated to EDI parsing I can
>>>give
>>> you some sneak preview of it. Here are some screenshots I made with
>>> upcoming Talend Open Studio 4.2.0:
>>>
>>> http://screencast.com/t/4mDhoHZw7
>>> http://screencast.com/t/rjijRpjB
>>> http://screencast.com/t/2Wlfp10ye
>>>
>>>
>>> UN/EDIFACT interchange support was introduced in Smooks 1.4, I don't
>>> think
>>> you could backport it easily to 1.2.4, however latest Talend components
>>> (tEDItoXML) are based on Smooks 1.4 therefore fully support UN/EDIFACT
>>> interchanges.
>>>
>>> Once your message is transformed to XML you could select any part that
>>> you
>>> are interested in.
>>>
>>> Concerning the specific separator char, I think it's doable, but AFAK
>>> separators are specified in UNH headers, I suppose they should be also
>>> treated well by 1.4 EDI Parser, but I might be wrong here. Anyway you
>>>can
>>> programmatically override the default separators.
>>>
>>> Best regards,
>>> Renat Zubairov
>>>
>>> --
>>> Product Owner ESB Tooling
>>> Talend Application Integration Division
>>>
>>>
>>>
>>>
>>>
>>> Am 28.04.11 22:14 schrieb "rmg78" unter <rmgiorgi@gmail.com>:
>>>
>>>>
>>>>Hi there,
>>>>I'm trying to use smooks in a talend flow... with some serious problems
>>>>and
>>>>I need some help:
>>>>
>>>>1) The smooks component for talend is based on an older version of
>>>>smooks
>>>>(1.2.4). Looking on the project site... it appears to be dead. Has
>anybody
>>>>tried to upgrade the component al least at version 1.4?
>>>>
>>>>2) The EDI files i'm trying to read have several types of messages.
>>>>I've
>>>>read that it's possible to manage this but in version 1.4
>>>>(interchange).
>>>>Two
>>>>questions about this: a)can i achieve this in version 1.2.4? b)i do not
>>>>need
>>>>to parse all the messages, just on or two types. For example... I've
>>>>the
>>>>following type of messages:
>>>>
>>>>{file begin}
>>>>        UNH+1+CSMPRR:08:1:1A' ... bla bla bla...
>>>>        UNH+1+BIDBRR:07:2:1A+999999'  ... bla bla bla...
>>>>        UNH+1+CCPRRR:09:4:1A+99999999'  ... bla bla bla...
>>>>        UNH+1+MHISRR:05:0:1A+999999'  ... bla bla bla...
>>>>{file end}
>>>>
>>>>I only want to process the CCPRRR message, who has inside... hundreds
>>>>of
>>>>items. I don't want to generate the mappings for the other types. Is it
>>>>possible?
>>>>
>>>>3) In the above example, i've converted the delimiters. In the real
>>>>file,
>>>>the delimiters are hexadecimal characters. Example: instead of plus
>>>>(+),
>>>>hay
>>>>have 1D (hex). Couldn't find to tell smooks config file to use this
>>>>kind
>>>>of
>>>>separators. Any ideas? i've files with more than 1Gb, so replace is not
>an
>>>>option.
>>>>
>>>>Enough for the moment, any help would be apreciated. The type of
>>>>messages
>>>>i'm using are from the air industry (amadeus - altea).
>>>>thanks
>>>>max.-
>>>>--
>>>>View this message in context:
>>>>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31500077.h
>>>>tml
>>>>Sent from the milyn - user mailing list archive at Nabble.com.
>>>>
>>>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe from this list, please visit:
>>>>
>>>>    http://xircles.codehaus.org/manage_email
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>     http://xircles.codehaus.org/manage_email
>>>
>>>
>>>
>>>
>>
>>
>
>--
>View this message in context:
>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31507923.html
>Sent from the milyn - user mailing list archive at Nabble.com.
>
>
>---------------------------------------------------------------------
>To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: smooks (EDI) + talend

Renat Zubairov-4
In reply to this post by rmg78
Hello Max,

Thnx for the info. Your feedback is very important to us, based on it we
will prioritize new features for the component, for example ability to
upload user-defined mappings :)

Best regards,
Renat Zubairov

--
Product Owner ESB Tooling
Talend Application Integration Division



Am 02.05.11 18:37 schrieb "rmg78" unter <[hidden email]>:

>In order to deploy the additional mappings i looked one of the existing
>jars, and based on that i replicated the content with my mappings.
>After that, i copied the jar in the talend component and in the lib/java
>folder and edited the component's files (*.properties and *.xml). The
>other
>thing i changed, was the configuration/ComponentsCache.javacache file,
>appending the new library.
>With these changes, I can see my "amadeus" package in the component's
>combo.
>
>I do not consider that these mappings should be integrated in TOS because
>they aren't standard mappings, the mappings are property of amadeus.
>
>In regard of TIS, i'll see it with my project manager.
>Thanks a lot for your support,
>max.-


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Ignoring unknown messages within one interchange

Renat Zubairov-4
In reply to this post by rmg78
* Moved discussion to the new subject (thread)

Hello Max,

Thank you for the description of your use-case, it's now much more clear
for me. It is a very important information for whole Smooks team.

Your sample is very interesting, usually we expect following structure:

 UNB #Interchange start
  UNH+1+XXX # Message start
  UNT # Message end
  UNH+2+XXX # Message start
  UNT # Message end

  ...
 UNZ #Interchange end

Apparently in your case you don't have the interchange header but only
individual messages, is it a specific variation of the UN/EDIFACT
standard? If it is a standard I think it would make sense for Smooks to
support it. If it is not a standard we could discuss on how to make our
parsing flexible so that Smooks users could declaratively (or with little
programming efforts) parse such structures.

Do you know wherever it's some specific standard behind this structure?


Best regards,
Renat Zubairov

--
Product Owner ESB Tooling
Talend Application Integration Division



Am 02.05.11 20:31 schrieb "rmg78" unter <[hidden email]>:

>Renat,
>The real life case is the following:
>
>1- I receive a tar-zipped file, with several .data files. These files are
>sent from the amadeus server directly to our filesystem.
>2- each .data file contains several edifact messages, being one of them
>the
>CCPRRR that i'm interested in. i need to discard the rest because i don't
>need them, and i don't have such mappings.
>
>As i explained, each file contains the messages as:
>
>Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1'
>Line 2: UNH+1+BIDBRR:07:2:1A+999999'  ... UNT+761+1'
>Line 3: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+762+1'
>Line 4: UNH+1+MHISRR:05:0:1A+999999'  ... UNT+763+1'
>Line 5: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+764+1'
>...
>With the component that i've now, if the file contains only CCPRRR
>messages
>it works fine, but in the above example... fails saying:
>
>Starting job EDIToXML_CCPRRR at 15:26 02/05/2011.
>
>[statistics] connecting to socket on port 3813
>[statistics] connected
>Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping model.
>Exception in component tFileOutputXML_1
>java.lang.NullPointerException
>    at
>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(EDI
>ToXML_CCPRRR.java:649)
>    at
>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CCPRR
>R.java:911)
>    at
>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.java:
>779)
>[statistics] disconnected
>Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1]
>
>What i don't want is to map every kind of message they sent in the files.
>any ideas?
>
>Thanks
>max


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Ignoring unknown messages within one interchange

rmg78
Renat,

See my comments below:
>is it a specific variation of the UN/EDIFACT standard?
>Do you know wherever it's some specific standard behind this structure?

I really don't know how to answer you, because the only thing we know is that this is the edifact standard for the whole air industry that use amadeus as partner (the other big player is sabre). But looking at the http://www.unece.org/trade/untdid/down_index.htm these messages aren't official. I think they have taken the protocol and implemented their own messages and kept it private. In fact, if you want the layout of these messages you must have access to their website to download them. :(
I would be very happy to send you one example... but this is production data and I can't publish such things on the forum. Anyway, i'll try to answer you and post all the data that i can. Tell me what do you need and i'll try to narrow the examples.

I'll append the 2 segments you say to see if the parser works fine discarding the messages i don't want. do you think it's going to work?
 
max.-

Renat Zubairov-4 wrote
* Moved discussion to the new subject (thread)

Hello Max,

Thank you for the description of your use-case, it's now much more clear
for me. It is a very important information for whole Smooks team.

Your sample is very interesting, usually we expect following structure:

 UNB #Interchange start
  UNH+1+XXX # Message start
  UNT # Message end
  UNH+2+XXX # Message start
  UNT # Message end

  ...
 UNZ #Interchange end

Apparently in your case you don't have the interchange header but only
individual messages, is it a specific variation of the UN/EDIFACT
standard? If it is a standard I think it would make sense for Smooks to
support it. If it is not a standard we could discuss on how to make our
parsing flexible so that Smooks users could declaratively (or with little
programming efforts) parse such structures.

Do you know wherever it's some specific standard behind this structure?


Best regards,
Renat Zubairov

--
Product Owner ESB Tooling
Talend Application Integration Division



Am 02.05.11 20:31 schrieb "rmg78" unter <rmgiorgi@gmail.com>:

>Renat,
>The real life case is the following:
>
>1- I receive a tar-zipped file, with several .data files. These files are
>sent from the amadeus server directly to our filesystem.
>2- each .data file contains several edifact messages, being one of them
>the
>CCPRRR that i'm interested in. i need to discard the rest because i don't
>need them, and i don't have such mappings.
>
>As i explained, each file contains the messages as:
>
>Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1'
>Line 2: UNH+1+BIDBRR:07:2:1A+999999'  ... UNT+761+1'
>Line 3: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+762+1'
>Line 4: UNH+1+MHISRR:05:0:1A+999999'  ... UNT+763+1'
>Line 5: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+764+1'
>...
>With the component that i've now, if the file contains only CCPRRR
>messages
>it works fine, but in the above example... fails saying:
>
>Starting job EDIToXML_CCPRRR at 15:26 02/05/2011.
>
>[statistics] connecting to socket on port 3813
>[statistics] connected
>Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping model.
>Exception in component tFileOutputXML_1
>java.lang.NullPointerException
>    at
>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(EDI
>ToXML_CCPRRR.java:649)
>    at
>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CCPRR
>R.java:911)
>    at
>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.java:
>779)
>[statistics] disconnected
>Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1]
>
>What i don't want is to map every kind of message they sent in the files.
>any ideas?
>
>Thanks
>max


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Ignoring unknown messages within one interchange

Tom Fennelly
In reply to this post by Renat Zubairov-4
Hey guys.... Trying to figure out what's going wrong.  I was thinking
about it and though Smooks should function OK with just the messages
(UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest.  It
worked fine with just the UNH and UNT segments (plus messages inside).

So is it just that we can't ignore unknown messages?

T.



On 02/05/2011 20:02, Renat Zubairov wrote:

> * Moved discussion to the new subject (thread)
>
> Hello Max,
>
> Thank you for the description of your use-case, it's now much more clear
> for me. It is a very important information for whole Smooks team.
>
> Your sample is very interesting, usually we expect following structure:
>
>   UNB #Interchange start
>    UNH+1+XXX # Message start
>    UNT # Message end
>    UNH+2+XXX # Message start
>    UNT # Message end
>
>    ...
>   UNZ #Interchange end
>
> Apparently in your case you don't have the interchange header but only
> individual messages, is it a specific variation of the UN/EDIFACT
> standard? If it is a standard I think it would make sense for Smooks to
> support it. If it is not a standard we could discuss on how to make our
> parsing flexible so that Smooks users could declaratively (or with little
> programming efforts) parse such structures.
>
> Do you know wherever it's some specific standard behind this structure?
>
>
> Best regards,
> Renat Zubairov
>
> --
> Product Owner ESB Tooling
> Talend Application Integration Division
>
>
>
> Am 02.05.11 20:31 schrieb "rmg78" unter<[hidden email]>:
>
>> Renat,
>> The real life case is the following:
>>
>> 1- I receive a tar-zipped file, with several .data files. These files are
>> sent from the amadeus server directly to our filesystem.
>> 2- each .data file contains several edifact messages, being one of them
>> the
>> CCPRRR that i'm interested in. i need to discard the rest because i don't
>> need them, and i don't have such mappings.
>>
>> As i explained, each file contains the messages as:
>>
>> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1'
>> Line 2: UNH+1+BIDBRR:07:2:1A+999999'  ... UNT+761+1'
>> Line 3: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+762+1'
>> Line 4: UNH+1+MHISRR:05:0:1A+999999'  ... UNT+763+1'
>> Line 5: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+764+1'
>> ...
>> With the component that i've now, if the file contains only CCPRRR
>> messages
>> it works fine, but in the above example... fails saying:
>>
>> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011.
>>
>> [statistics] connecting to socket on port 3813
>> [statistics] connected
>> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping model.
>> Exception in component tFileOutputXML_1
>> java.lang.NullPointerException
>>     at
>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(EDI
>> ToXML_CCPRRR.java:649)
>>     at
>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CCPRR
>> R.java:911)
>>     at
>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.java:
>> 779)
>> [statistics] disconnected
>> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1]
>>
>> What i don't want is to map every kind of message they sent in the files.
>> any ideas?
>>
>> Thanks
>> max
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>      http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Ignoring unknown messages within one interchange

rmg78
Hi Tom,
Exactly, it's all about ignoring unknown (or unmapped) messages, the rest seems to be working fine despite i couldn´t test the complete files (because of the mixed messages).
Maybe there could be a flag to tell smooks to be strict (fail if exists unmapped messages, the actual behavior) or not to be strict (ignore the unmapped messages). Do you think this could be done?

thanks in advance
max.-

TomFennelly wrote
Hey guys.... Trying to figure out what's going wrong.  I was thinking
about it and though Smooks should function OK with just the messages
(UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest.  It
worked fine with just the UNH and UNT segments (plus messages inside).

So is it just that we can't ignore unknown messages?

T.



On 02/05/2011 20:02, Renat Zubairov wrote:
> * Moved discussion to the new subject (thread)
>
> Hello Max,
>
> Thank you for the description of your use-case, it's now much more clear
> for me. It is a very important information for whole Smooks team.
>
> Your sample is very interesting, usually we expect following structure:
>
>   UNB #Interchange start
>    UNH+1+XXX # Message start
>    UNT # Message end
>    UNH+2+XXX # Message start
>    UNT # Message end
>
>    ...
>   UNZ #Interchange end
>
> Apparently in your case you don't have the interchange header but only
> individual messages, is it a specific variation of the UN/EDIFACT
> standard? If it is a standard I think it would make sense for Smooks to
> support it. If it is not a standard we could discuss on how to make our
> parsing flexible so that Smooks users could declaratively (or with little
> programming efforts) parse such structures.
>
> Do you know wherever it's some specific standard behind this structure?
>
>
> Best regards,
> Renat Zubairov
>
> --
> Product Owner ESB Tooling
> Talend Application Integration Division
>
>
>
> Am 02.05.11 20:31 schrieb "rmg78" unter<rmgiorgi@gmail.com>:
>
>> Renat,
>> The real life case is the following:
>>
>> 1- I receive a tar-zipped file, with several .data files. These files are
>> sent from the amadeus server directly to our filesystem.
>> 2- each .data file contains several edifact messages, being one of them
>> the
>> CCPRRR that i'm interested in. i need to discard the rest because i don't
>> need them, and i don't have such mappings.
>>
>> As i explained, each file contains the messages as:
>>
>> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1'
>> Line 2: UNH+1+BIDBRR:07:2:1A+999999'  ... UNT+761+1'
>> Line 3: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+762+1'
>> Line 4: UNH+1+MHISRR:05:0:1A+999999'  ... UNT+763+1'
>> Line 5: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+764+1'
>> ...
>> With the component that i've now, if the file contains only CCPRRR
>> messages
>> it works fine, but in the above example... fails saying:
>>
>> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011.
>>
>> [statistics] connecting to socket on port 3813
>> [statistics] connected
>> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping model.
>> Exception in component tFileOutputXML_1
>> java.lang.NullPointerException
>>     at
>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(EDI
>> ToXML_CCPRRR.java:649)
>>     at
>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CCPRR
>> R.java:911)
>>     at
>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.java:
>> 779)
>> [statistics] disconnected
>> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1]
>>
>> What i don't want is to map every kind of message they sent in the files.
>> any ideas?
>>
>> Thanks
>> max
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>      http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Ignoring unknown messages within one interchange

Renat Zubairov-4
Hello Max, Tom,

IMO ignoring message types where we can't find mapping information could
be a good feature candidate, however with 1.5 release we have a
significant difference to 1.4. In 1.4 you would need to declare explicit
list of mappings available for UN/EDIFACT parser, in 1.5 we load them
lazily based on the information we could find in the classpath.

@Max: can you create a JIRA issue. We could use a specific FEATURE flag
for the UN/EDIFACT parser just like we do with ignore new-lines.
@Tom: what do you think?

Best regards,
Renat Zubairov



Am 03.05.11 03:00 schrieb "rmg78" unter <[hidden email]>:

>
>Hi Tom,
>Exactly, it's all about ignoring unknown (or unmapped) messages, the rest
>seems to be working fine despite i couldn´t test the complete files
>(because
>of the mixed messages).
>Maybe there could be a flag to tell smooks to be strict (fail if exists
>unmapped messages, the actual behavior) or not to be strict (ignore the
>unmapped messages). Do you think this could be done?
>
>thanks in advance
>max.-
>
>
>TomFennelly wrote:
>>
>> Hey guys.... Trying to figure out what's going wrong.  I was thinking
>> about it and though Smooks should function OK with just the messages
>> (UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest.  It
>> worked fine with just the UNH and UNT segments (plus messages inside).
>>
>> So is it just that we can't ignore unknown messages?
>>
>> T.
>>
>>
>>
>> On 02/05/2011 20:02, Renat Zubairov wrote:
>>> * Moved discussion to the new subject (thread)
>>>
>>> Hello Max,
>>>
>>> Thank you for the description of your use-case, it's now much more
>>>clear
>>> for me. It is a very important information for whole Smooks team.
>>>
>>> Your sample is very interesting, usually we expect following structure:
>>>
>>>   UNB #Interchange start
>>>    UNH+1+XXX # Message start
>>>    UNT # Message end
>>>    UNH+2+XXX # Message start
>>>    UNT # Message end
>>>
>>>    ...
>>>   UNZ #Interchange end
>>>
>>> Apparently in your case you don't have the interchange header but only
>>> individual messages, is it a specific variation of the UN/EDIFACT
>>> standard? If it is a standard I think it would make sense for Smooks to
>>> support it. If it is not a standard we could discuss on how to make our
>>> parsing flexible so that Smooks users could declaratively (or with
>>>little
>>> programming efforts) parse such structures.
>>>
>>> Do you know wherever it's some specific standard behind this structure?
>>>
>>>
>>> Best regards,
>>> Renat Zubairov
>>>
>>> --
>>> Product Owner ESB Tooling
>>> Talend Application Integration Division
>>>
>>>
>>>
>>> Am 02.05.11 20:31 schrieb "rmg78" unter<[hidden email]>:
>>>
>>>> Renat,
>>>> The real life case is the following:
>>>>
>>>> 1- I receive a tar-zipped file, with several .data files. These files
>>>> are
>>>> sent from the amadeus server directly to our filesystem.
>>>> 2- each .data file contains several edifact messages, being one of
>>>>them
>>>> the
>>>> CCPRRR that i'm interested in. i need to discard the rest because i
>>>> don't
>>>> need them, and i don't have such mappings.
>>>>
>>>> As i explained, each file contains the messages as:
>>>>
>>>> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1'
>>>> Line 2: UNH+1+BIDBRR:07:2:1A+999999'  ... UNT+761+1'
>>>> Line 3: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+762+1'
>>>> Line 4: UNH+1+MHISRR:05:0:1A+999999'  ... UNT+763+1'
>>>> Line 5: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+764+1'
>>>> ...
>>>> With the component that i've now, if the file contains only CCPRRR
>>>> messages
>>>> it works fine, but in the above example... fails saying:
>>>>
>>>> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011.
>>>>
>>>> [statistics] connecting to socket on port 3813
>>>> [statistics] connected
>>>> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping
>>>> model.
>>>> Exception in component tFileOutputXML_1
>>>> java.lang.NullPointerException
>>>>     at
>>>>
>>>>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(
>>>>EDI
>>>> ToXML_CCPRRR.java:649)
>>>>     at
>>>>
>>>>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CC
>>>>PRR
>>>> R.java:911)
>>>>     at
>>>>
>>>>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.ja
>>>>va:
>>>> 779)
>>>> [statistics] disconnected
>>>> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1]
>>>>
>>>> What i don't want is to map every kind of message they sent in the
>>>> files.
>>>> any ideas?
>>>>
>>>> Thanks
>>>> max
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>      http://xircles.codehaus.org/manage_email
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>>
>>
>
>--
>View this message in context:
>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31528727.html
>Sent from the milyn - user mailing list archive at Nabble.com.
>
>
>---------------------------------------------------------------------
>To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Ignoring unknown messages within one interchange

Tom Fennelly
Hey guys.

Yes... I'd be cool with adding that.  I'll create a JIRA for it.

T.


On 03/05/2011 07:45, Renat Zubairov wrote:

> Hello Max, Tom,
>
> IMO ignoring message types where we can't find mapping information could
> be a good feature candidate, however with 1.5 release we have a
> significant difference to 1.4. In 1.4 you would need to declare explicit
> list of mappings available for UN/EDIFACT parser, in 1.5 we load them
> lazily based on the information we could find in the classpath.
>
> @Max: can you create a JIRA issue. We could use a specific FEATURE flag
> for the UN/EDIFACT parser just like we do with ignore new-lines.
> @Tom: what do you think?
>
> Best regards,
> Renat Zubairov
>
>
>
> Am 03.05.11 03:00 schrieb "rmg78" unter<[hidden email]>:
>
>> Hi Tom,
>> Exactly, it's all about ignoring unknown (or unmapped) messages, the rest
>> seems to be working fine despite i couldn´t test the complete files
>> (because
>> of the mixed messages).
>> Maybe there could be a flag to tell smooks to be strict (fail if exists
>> unmapped messages, the actual behavior) or not to be strict (ignore the
>> unmapped messages). Do you think this could be done?
>>
>> thanks in advance
>> max.-
>>
>>
>> TomFennelly wrote:
>>> Hey guys.... Trying to figure out what's going wrong.  I was thinking
>>> about it and though Smooks should function OK with just the messages
>>> (UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest.  It
>>> worked fine with just the UNH and UNT segments (plus messages inside).
>>>
>>> So is it just that we can't ignore unknown messages?
>>>
>>> T.
>>>
>>>
>>>
>>> On 02/05/2011 20:02, Renat Zubairov wrote:
>>>> * Moved discussion to the new subject (thread)
>>>>
>>>> Hello Max,
>>>>
>>>> Thank you for the description of your use-case, it's now much more
>>>> clear
>>>> for me. It is a very important information for whole Smooks team.
>>>>
>>>> Your sample is very interesting, usually we expect following structure:
>>>>
>>>>    UNB #Interchange start
>>>>     UNH+1+XXX # Message start
>>>>     UNT # Message end
>>>>     UNH+2+XXX # Message start
>>>>     UNT # Message end
>>>>
>>>>     ...
>>>>    UNZ #Interchange end
>>>>
>>>> Apparently in your case you don't have the interchange header but only
>>>> individual messages, is it a specific variation of the UN/EDIFACT
>>>> standard? If it is a standard I think it would make sense for Smooks to
>>>> support it. If it is not a standard we could discuss on how to make our
>>>> parsing flexible so that Smooks users could declaratively (or with
>>>> little
>>>> programming efforts) parse such structures.
>>>>
>>>> Do you know wherever it's some specific standard behind this structure?
>>>>
>>>>
>>>> Best regards,
>>>> Renat Zubairov
>>>>
>>>> --
>>>> Product Owner ESB Tooling
>>>> Talend Application Integration Division
>>>>
>>>>
>>>>
>>>> Am 02.05.11 20:31 schrieb "rmg78" unter<[hidden email]>:
>>>>
>>>>> Renat,
>>>>> The real life case is the following:
>>>>>
>>>>> 1- I receive a tar-zipped file, with several .data files. These files
>>>>> are
>>>>> sent from the amadeus server directly to our filesystem.
>>>>> 2- each .data file contains several edifact messages, being one of
>>>>> them
>>>>> the
>>>>> CCPRRR that i'm interested in. i need to discard the rest because i
>>>>> don't
>>>>> need them, and i don't have such mappings.
>>>>>
>>>>> As i explained, each file contains the messages as:
>>>>>
>>>>> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1'
>>>>> Line 2: UNH+1+BIDBRR:07:2:1A+999999'  ... UNT+761+1'
>>>>> Line 3: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+762+1'
>>>>> Line 4: UNH+1+MHISRR:05:0:1A+999999'  ... UNT+763+1'
>>>>> Line 5: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+764+1'
>>>>> ...
>>>>> With the component that i've now, if the file contains only CCPRRR
>>>>> messages
>>>>> it works fine, but in the above example... fails saying:
>>>>>
>>>>> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011.
>>>>>
>>>>> [statistics] connecting to socket on port 3813
>>>>> [statistics] connected
>>>>> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping
>>>>> model.
>>>>> Exception in component tFileOutputXML_1
>>>>> java.lang.NullPointerException
>>>>>      at
>>>>>
>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(
>>>>> EDI
>>>>> ToXML_CCPRRR.java:649)
>>>>>      at
>>>>>
>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CC
>>>>> PRR
>>>>> R.java:911)
>>>>>      at
>>>>>
>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.ja
>>>>> va:
>>>>> 779)
>>>>> [statistics] disconnected
>>>>> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1]
>>>>>
>>>>> What i don't want is to map every kind of message they sent in the
>>>>> files.
>>>>> any ideas?
>>>>>
>>>>> Thanks
>>>>> max
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe from this list, please visit:
>>>>
>>>>       http://xircles.codehaus.org/manage_email
>>>>
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>      http://xircles.codehaus.org/manage_email
>>>
>>>
>>>
>>>
>> --
>> View this message in context:
>> http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31528727.html
>> Sent from the milyn - user mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>      http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Ignoring unknown messages within one interchange

Tom Fennelly
http://jira.codehaus.org/browse/MILYN-602

On 03/05/2011 10:08, Tom Fennelly wrote:

> Hey guys.
>
> Yes... I'd be cool with adding that.  I'll create a JIRA for it.
>
> T.
>
>
> On 03/05/2011 07:45, Renat Zubairov wrote:
>> Hello Max, Tom,
>>
>> IMO ignoring message types where we can't find mapping information could
>> be a good feature candidate, however with 1.5 release we have a
>> significant difference to 1.4. In 1.4 you would need to declare explicit
>> list of mappings available for UN/EDIFACT parser, in 1.5 we load them
>> lazily based on the information we could find in the classpath.
>>
>> @Max: can you create a JIRA issue. We could use a specific FEATURE flag
>> for the UN/EDIFACT parser just like we do with ignore new-lines.
>> @Tom: what do you think?
>>
>> Best regards,
>> Renat Zubairov
>>
>>
>>
>> Am 03.05.11 03:00 schrieb "rmg78" unter<[hidden email]>:
>>
>>> Hi Tom,
>>> Exactly, it's all about ignoring unknown (or unmapped) messages, the
>>> rest
>>> seems to be working fine despite i couldn´t test the complete files
>>> (because
>>> of the mixed messages).
>>> Maybe there could be a flag to tell smooks to be strict (fail if exists
>>> unmapped messages, the actual behavior) or not to be strict (ignore the
>>> unmapped messages). Do you think this could be done?
>>>
>>> thanks in advance
>>> max.-
>>>
>>>
>>> TomFennelly wrote:
>>>> Hey guys.... Trying to figure out what's going wrong.  I was thinking
>>>> about it and though Smooks should function OK with just the messages
>>>> (UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest.  It
>>>> worked fine with just the UNH and UNT segments (plus messages inside).
>>>>
>>>> So is it just that we can't ignore unknown messages?
>>>>
>>>> T.
>>>>
>>>>
>>>>
>>>> On 02/05/2011 20:02, Renat Zubairov wrote:
>>>>> * Moved discussion to the new subject (thread)
>>>>>
>>>>> Hello Max,
>>>>>
>>>>> Thank you for the description of your use-case, it's now much more
>>>>> clear
>>>>> for me. It is a very important information for whole Smooks team.
>>>>>
>>>>> Your sample is very interesting, usually we expect following
>>>>> structure:
>>>>>
>>>>>    UNB #Interchange start
>>>>>     UNH+1+XXX # Message start
>>>>>     UNT # Message end
>>>>>     UNH+2+XXX # Message start
>>>>>     UNT # Message end
>>>>>
>>>>>     ...
>>>>>    UNZ #Interchange end
>>>>>
>>>>> Apparently in your case you don't have the interchange header but
>>>>> only
>>>>> individual messages, is it a specific variation of the UN/EDIFACT
>>>>> standard? If it is a standard I think it would make sense for
>>>>> Smooks to
>>>>> support it. If it is not a standard we could discuss on how to
>>>>> make our
>>>>> parsing flexible so that Smooks users could declaratively (or with
>>>>> little
>>>>> programming efforts) parse such structures.
>>>>>
>>>>> Do you know wherever it's some specific standard behind this
>>>>> structure?
>>>>>
>>>>>
>>>>> Best regards,
>>>>> Renat Zubairov
>>>>>
>>>>> --
>>>>> Product Owner ESB Tooling
>>>>> Talend Application Integration Division
>>>>>
>>>>>
>>>>>
>>>>> Am 02.05.11 20:31 schrieb "rmg78" unter<[hidden email]>:
>>>>>
>>>>>> Renat,
>>>>>> The real life case is the following:
>>>>>>
>>>>>> 1- I receive a tar-zipped file, with several .data files. These
>>>>>> files
>>>>>> are
>>>>>> sent from the amadeus server directly to our filesystem.
>>>>>> 2- each .data file contains several edifact messages, being one of
>>>>>> them
>>>>>> the
>>>>>> CCPRRR that i'm interested in. i need to discard the rest because i
>>>>>> don't
>>>>>> need them, and i don't have such mappings.
>>>>>>
>>>>>> As i explained, each file contains the messages as:
>>>>>>
>>>>>> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1'
>>>>>> Line 2: UNH+1+BIDBRR:07:2:1A+999999'  ... UNT+761+1'
>>>>>> Line 3: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+762+1'
>>>>>> Line 4: UNH+1+MHISRR:05:0:1A+999999'  ... UNT+763+1'
>>>>>> Line 5: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+764+1'
>>>>>> ...
>>>>>> With the component that i've now, if the file contains only CCPRRR
>>>>>> messages
>>>>>> it works fine, but in the above example... fails saying:
>>>>>>
>>>>>> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011.
>>>>>>
>>>>>> [statistics] connecting to socket on port 3813
>>>>>> [statistics] connected
>>>>>> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping
>>>>>> model.
>>>>>> Exception in component tFileOutputXML_1
>>>>>> java.lang.NullPointerException
>>>>>>      at
>>>>>>
>>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(
>>>>>>
>>>>>> EDI
>>>>>> ToXML_CCPRRR.java:649)
>>>>>>      at
>>>>>>
>>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CC
>>>>>>
>>>>>> PRR
>>>>>> R.java:911)
>>>>>>      at
>>>>>>
>>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.ja
>>>>>>
>>>>>> va:
>>>>>> 779)
>>>>>> [statistics] disconnected
>>>>>> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1]
>>>>>>
>>>>>> What i don't want is to map every kind of message they sent in the
>>>>>> files.
>>>>>> any ideas?
>>>>>>
>>>>>> Thanks
>>>>>> max
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe from this list, please visit:
>>>>>
>>>>>       http://xircles.codehaus.org/manage_email
>>>>>
>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe from this list, please visit:
>>>>
>>>>      http://xircles.codehaus.org/manage_email
>>>>
>>>>
>>>>
>>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31528727.html 
>>>
>>> Sent from the milyn - user mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>     http://xircles.codehaus.org/manage_email
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>      http://xircles.codehaus.org/manage_email
>>
>>
>>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Ignoring unknown messages within one interchange

rmg78
Thanks Tom, it's good to know about the new feature.
Do you think it's possible to have a patch for the 1.4 version? I think Talend 4.2 will be available soon but with smooks 1.4. I don't know the roadmap for having smooks 1.5 in Talend... but i think will be months right?

thanks for your support
max.-

TomFennelly wrote
http://jira.codehaus.org/browse/MILYN-602

On 03/05/2011 10:08, Tom Fennelly wrote:
> Hey guys.
>
> Yes... I'd be cool with adding that.  I'll create a JIRA for it.
>
> T.
>
>
> On 03/05/2011 07:45, Renat Zubairov wrote:
>> Hello Max, Tom,
>>
>> IMO ignoring message types where we can't find mapping information could
>> be a good feature candidate, however with 1.5 release we have a
>> significant difference to 1.4. In 1.4 you would need to declare explicit
>> list of mappings available for UN/EDIFACT parser, in 1.5 we load them
>> lazily based on the information we could find in the classpath.
>>
>> @Max: can you create a JIRA issue. We could use a specific FEATURE flag
>> for the UN/EDIFACT parser just like we do with ignore new-lines.
>> @Tom: what do you think?
>>
>> Best regards,
>> Renat Zubairov
>>
>>
>>
>> Am 03.05.11 03:00 schrieb "rmg78" unter<rmgiorgi@gmail.com>:
>>
>>> Hi Tom,
>>> Exactly, it's all about ignoring unknown (or unmapped) messages, the
>>> rest
>>> seems to be working fine despite i couldn´t test the complete files
>>> (because
>>> of the mixed messages).
>>> Maybe there could be a flag to tell smooks to be strict (fail if exists
>>> unmapped messages, the actual behavior) or not to be strict (ignore the
>>> unmapped messages). Do you think this could be done?
>>>
>>> thanks in advance
>>> max.-
>>>
>>>
>>> TomFennelly wrote:
>>>> Hey guys.... Trying to figure out what's going wrong.  I was thinking
>>>> about it and though Smooks should function OK with just the messages
>>>> (UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest.  It
>>>> worked fine with just the UNH and UNT segments (plus messages inside).
>>>>
>>>> So is it just that we can't ignore unknown messages?
>>>>
>>>> T.
>>>>
>>>>
>>>>
>>>> On 02/05/2011 20:02, Renat Zubairov wrote:
>>>>> * Moved discussion to the new subject (thread)
>>>>>
>>>>> Hello Max,
>>>>>
>>>>> Thank you for the description of your use-case, it's now much more
>>>>> clear
>>>>> for me. It is a very important information for whole Smooks team.
>>>>>
>>>>> Your sample is very interesting, usually we expect following
>>>>> structure:
>>>>>
>>>>>    UNB #Interchange start
>>>>>     UNH+1+XXX # Message start
>>>>>     UNT # Message end
>>>>>     UNH+2+XXX # Message start
>>>>>     UNT # Message end
>>>>>
>>>>>     ...
>>>>>    UNZ #Interchange end
>>>>>
>>>>> Apparently in your case you don't have the interchange header but
>>>>> only
>>>>> individual messages, is it a specific variation of the UN/EDIFACT
>>>>> standard? If it is a standard I think it would make sense for
>>>>> Smooks to
>>>>> support it. If it is not a standard we could discuss on how to
>>>>> make our
>>>>> parsing flexible so that Smooks users could declaratively (or with
>>>>> little
>>>>> programming efforts) parse such structures.
>>>>>
>>>>> Do you know wherever it's some specific standard behind this
>>>>> structure?
>>>>>
>>>>>
>>>>> Best regards,
>>>>> Renat Zubairov
>>>>>
>>>>> --
>>>>> Product Owner ESB Tooling
>>>>> Talend Application Integration Division
>>>>>
>>>>>
>>>>>
>>>>> Am 02.05.11 20:31 schrieb "rmg78" unter<rmgiorgi@gmail.com>:
>>>>>
>>>>>> Renat,
>>>>>> The real life case is the following:
>>>>>>
>>>>>> 1- I receive a tar-zipped file, with several .data files. These
>>>>>> files
>>>>>> are
>>>>>> sent from the amadeus server directly to our filesystem.
>>>>>> 2- each .data file contains several edifact messages, being one of
>>>>>> them
>>>>>> the
>>>>>> CCPRRR that i'm interested in. i need to discard the rest because i
>>>>>> don't
>>>>>> need them, and i don't have such mappings.
>>>>>>
>>>>>> As i explained, each file contains the messages as:
>>>>>>
>>>>>> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1'
>>>>>> Line 2: UNH+1+BIDBRR:07:2:1A+999999'  ... UNT+761+1'
>>>>>> Line 3: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+762+1'
>>>>>> Line 4: UNH+1+MHISRR:05:0:1A+999999'  ... UNT+763+1'
>>>>>> Line 5: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+764+1'
>>>>>> ...
>>>>>> With the component that i've now, if the file contains only CCPRRR
>>>>>> messages
>>>>>> it works fine, but in the above example... fails saying:
>>>>>>
>>>>>> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011.
>>>>>>
>>>>>> [statistics] connecting to socket on port 3813
>>>>>> [statistics] connected
>>>>>> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping
>>>>>> model.
>>>>>> Exception in component tFileOutputXML_1
>>>>>> java.lang.NullPointerException
>>>>>>      at
>>>>>>
>>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(
>>>>>>
>>>>>> EDI
>>>>>> ToXML_CCPRRR.java:649)
>>>>>>      at
>>>>>>
>>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CC
>>>>>>
>>>>>> PRR
>>>>>> R.java:911)
>>>>>>      at
>>>>>>
>>>>>> aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.ja
>>>>>>
>>>>>> va:
>>>>>> 779)
>>>>>> [statistics] disconnected
>>>>>> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1]
>>>>>>
>>>>>> What i don't want is to map every kind of message they sent in the
>>>>>> files.
>>>>>> any ideas?
>>>>>>
>>>>>> Thanks
>>>>>> max
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe from this list, please visit:
>>>>>
>>>>>       http://xircles.codehaus.org/manage_email
>>>>>
>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe from this list, please visit:
>>>>
>>>>      http://xircles.codehaus.org/manage_email
>>>>
>>>>
>>>>
>>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31528727.html 
>>>
>>> Sent from the milyn - user mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>     http://xircles.codehaus.org/manage_email
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>      http://xircles.codehaus.org/manage_email
>>
>>
>>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: Ignoring unknown messages within one interchange

rmg78
In reply to this post by Renat Zubairov-4
Hi Renat,

Tom posted the new Jira issue with the feature request (excellent). I didn't understand the problem  about smooks 1.5 and the classpath, but it seems that having a flag for ignoring unmapped messages it's a good idea.
The feature it's proposed for smooks 1.5, do you have an idea of when it's going to be available in Talend?
thanks
max.-  

Renat Zubairov-4 wrote
Hello Max, Tom,

IMO ignoring message types where we can't find mapping information could
be a good feature candidate, however with 1.5 release we have a
significant difference to 1.4. In 1.4 you would need to declare explicit
list of mappings available for UN/EDIFACT parser, in 1.5 we load them
lazily based on the information we could find in the classpath.

@Max: can you create a JIRA issue. We could use a specific FEATURE flag
for the UN/EDIFACT parser just like we do with ignore new-lines.
@Tom: what do you think?

Best regards,
Renat Zubairov



Am 03.05.11 03:00 schrieb "rmg78" unter <rmgiorgi@gmail.com>:

>
>Hi Tom,
>Exactly, it's all about ignoring unknown (or unmapped) messages, the rest
>seems to be working fine despite i couldn´t test the complete files
>(because
>of the mixed messages).
>Maybe there could be a flag to tell smooks to be strict (fail if exists
>unmapped messages, the actual behavior) or not to be strict (ignore the
>unmapped messages). Do you think this could be done?
>
>thanks in advance
>max.-
>
>
>TomFennelly wrote:
>>
>> Hey guys.... Trying to figure out what's going wrong.  I was thinking
>> about it and though Smooks should function OK with just the messages
>> (UNH/UNT) so I tried it out by modifying the UNEdifactReaderTest.  It
>> worked fine with just the UNH and UNT segments (plus messages inside).
>>
>> So is it just that we can't ignore unknown messages?
>>
>> T.
>>
>>
>>
>> On 02/05/2011 20:02, Renat Zubairov wrote:
>>> * Moved discussion to the new subject (thread)
>>>
>>> Hello Max,
>>>
>>> Thank you for the description of your use-case, it's now much more
>>>clear
>>> for me. It is a very important information for whole Smooks team.
>>>
>>> Your sample is very interesting, usually we expect following structure:
>>>
>>>   UNB #Interchange start
>>>    UNH+1+XXX # Message start
>>>    UNT # Message end
>>>    UNH+2+XXX # Message start
>>>    UNT # Message end
>>>
>>>    ...
>>>   UNZ #Interchange end
>>>
>>> Apparently in your case you don't have the interchange header but only
>>> individual messages, is it a specific variation of the UN/EDIFACT
>>> standard? If it is a standard I think it would make sense for Smooks to
>>> support it. If it is not a standard we could discuss on how to make our
>>> parsing flexible so that Smooks users could declaratively (or with
>>>little
>>> programming efforts) parse such structures.
>>>
>>> Do you know wherever it's some specific standard behind this structure?
>>>
>>>
>>> Best regards,
>>> Renat Zubairov
>>>
>>> --
>>> Product Owner ESB Tooling
>>> Talend Application Integration Division
>>>
>>>
>>>
>>> Am 02.05.11 20:31 schrieb "rmg78" unter<rmgiorgi@gmail.com>:
>>>
>>>> Renat,
>>>> The real life case is the following:
>>>>
>>>> 1- I receive a tar-zipped file, with several .data files. These files
>>>> are
>>>> sent from the amadeus server directly to our filesystem.
>>>> 2- each .data file contains several edifact messages, being one of
>>>>them
>>>> the
>>>> CCPRRR that i'm interested in. i need to discard the rest because i
>>>> don't
>>>> need them, and i don't have such mappings.
>>>>
>>>> As i explained, each file contains the messages as:
>>>>
>>>> Line 1: UNH+1+CSMPRR:08:1:1A' ... UNT+760+1'
>>>> Line 2: UNH+1+BIDBRR:07:2:1A+999999'  ... UNT+761+1'
>>>> Line 3: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+762+1'
>>>> Line 4: UNH+1+MHISRR:05:0:1A+999999'  ... UNT+763+1'
>>>> Line 5: UNH+1+CCPRRR:09:4:1A+99999999'  ... UNT+764+1'
>>>> ...
>>>> With the component that i've now, if the file contains only CCPRRR
>>>> messages
>>>> it works fine, but in the above example... fails saying:
>>>>
>>>> Starting job EDIToXML_CCPRRR at 15:26 02/05/2011.
>>>>
>>>> [statistics] connecting to socket on port 3813
>>>> [statistics] connected
>>>> Mapping Model 'CSMPRR:08:1:1A' not found in supplied set of Mapping
>>>> model.
>>>> Exception in component tFileOutputXML_1
>>>> java.lang.NullPointerException
>>>>     at
>>>>
>>>>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.tEDIFACTtoXML_1Process(
>>>>EDI
>>>> ToXML_CCPRRR.java:649)
>>>>     at
>>>>
>>>>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.runJobInTOS(EDIToXML_CC
>>>>PRR
>>>> R.java:911)
>>>>     at
>>>>
>>>>aracsfx_old.editoxml_ccprrr_0_1.EDIToXML_CCPRRR.main(EDIToXML_CCPRRR.ja
>>>>va:
>>>> 779)
>>>> [statistics] disconnected
>>>> Job EDIToXML_CCPRRR ended at 15:26 02/05/2011. [exit code=1]
>>>>
>>>> What i don't want is to map every kind of message they sent in the
>>>> files.
>>>> any ideas?
>>>>
>>>> Thanks
>>>> max
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>      http://xircles.codehaus.org/manage_email
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>>
>>
>
>--
>View this message in context:
>http://old.nabble.com/smooks-%28EDI%29-%2B-talend-tp31500077p31528727.html
>Sent from the milyn - user mailing list archive at Nabble.com.
>
>
>---------------------------------------------------------------------
>To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: smooks (EDI) + talend

mabchour1989
In reply to this post by rmg78
Hey,

I actually do work on the similar edifact syntax and I know it's private to Amadeus, but still i would like to ask you if you can provide me some documentation or immplementation guide that shows this syntax and its meaning I would really appreciate it. I'm working on a governemental project that involves air industry.

my email address is mo.mabchour@gmail.com

Thank you man, reply to this message in my email not this forum.
Reply | Threaded
Open this post in threaded view
|

Re: smooks (EDI) + talend

pmrsmilyn
This post has NOT been accepted by the mailing list yet.
In reply to this post by rmg78
Hi Max, I am trying to use the tsmook to read CCPRRR and MHISRR EDIFACT messages from Amadeus for TAP. Can you provide the XML mapping or is that must to ask? Can you explain what are the pain points? Thanks, Pedro
12