You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ofbiz.apache.org by James Lawton <ja...@tofinosoftware.com> on 2010/12/01 23:36:00 UTC
RE: [SPAM]Re: How to import a pre existing product database
Hmmm, not sure what happened there - I had appended the logs and a snapshot of the working with files screen...
Is there something which strips out attachments?
I'll try again
-----Original Message-----
From: BJ Freeman [mailto:bjfree@free-man.net]
Sent: December-01-10 3:41 PM
To: user@ofbiz.apache.org
Subject: [SPAM]Re: How to import a pre existing product database
sorry see no logs and not enough detail.
I have no way to replicated your steps.
=========================
BJ Freeman
Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
Chat Y! messenger: bjfr33man
James Lawton sent the following on 12/1/2010 12:23 PM:
> Here is the section of ofbiz.log for the timeframe
>
>
>
>
>
>
>
>
>
> Here is what I did:
>
> entered the file information and pressed "Run"
>
>
>
>
>
>
>
>
>
> Thanks
>
> James
>
>
>
>
>
> -----Original Message-----
> From: BJ Freeman [mailto:bjfree@free-man.net]
> Sent: December-01-10 2:18 PM
> To: user@ofbiz.apache.org
> Subject: [SPAM]Re: How to import a pre existing product database
>
>
>
> take a look at your logs.
>
> please give detail steps
>
>
>
>
>
> =========================
>
> BJ Freeman
>
> Strategic Power Office with Supplier Automation<
> <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>
> Specialtymarket.com< <http://www.specialtymarket.com/>
> http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
>
>
> Chat Y! messenger: bjfr33man
>
>
>
>
>
> James Lawton sent the following on 12/1/2010 10:27 AM:
>
>> Thanks for the link and the record information...
>
>>
>
>> I have altered the fields to fit the product record field names and
>
>> tried to use the "work with data files" tool again
>
>>
>
>> I enter the Definition File Name in the screen
>
>> "<file:///\\locahost\c:\files\tofino\import%20data\tofinodata.xml> file://locahost/c:\files\tofino\import data\tofinodata.xml" and select the URL check box and then hit Run. The screen flickers but nothing happens. The instructions say that a drop down box will appear with a list of the definitions to pick from...
>
>>
>
>> Thoughts?
>
>>
>
>> Here is the modified definition file:
>
>>
>
>> <data-file name="c:\files\tofino\import data\itemdatabasereportmod.txt" separator-style="delimited" type-code="UTF-8"
>
>> delimiter="	" text-delimiter=""
>
>> description="this takes the product data from jeffs file"
>
>> has-header="true">
>
>> <record name="Product">
>
>> <field name="manufacturerPartyId" ignored="false"
>> type="String"/>
>
>> <field name="productId" ignored="false" type="String"/>
>
>> <field name="primaryProductCategoryId" expression="true"
>
>> default-value="tools" type="String"/>
>
>> <field name="productTypeId" expression="true"
>
>> default-value="FINISHED_GOOD" type="String"/>
>
>> <field name="productName" ignored="false" type="String"/>
>
>> <field name="description" ignored="false" type="String"/>
>
>> <field name="quantityIncluded" expression="true" default-value="1"
>
>> type="String"/>
>
>> <field name="priceDetailText" ignored="true"
>> type="String"/>
>
>> </record>
>
>> </data-file>
>
>>
>
>>
>
>> Thanks
>
>> James
>
>> -----Original Message-----
>
>> From: BJ Freeman [mailto:bjfree@free-man.net]
>
>> Sent: December-01-10 12:37 PM
>
>> To: user@ofbiz.apache.org
>
>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>
>
>> if you look at the table the manufacture field is in then look up the
>
>> same in the entities
>
>> <https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint>
>> https://demo-trunk.ofbiz.apache.org/webtools/control/entitymaint
>
>> Just to clarify
>
>> your file has a Record=Product.
>
>> this means you will be creating a Product entity file, so only fields in the product entity can be created.
>
>>
>> <https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?ent
>> it>
>> https://demo-trunk.ofbiz.apache.org/webtools/control/FindGeneric?enti
>> t
>
>> yName=Product&find=true&VIEW_SIZE=50&VIEW_INDEX=0
>
>> if you have data that goes in different entities then you need to create a Record for each entity with its fields.
>
>>
>
>>
>
>> James Lawton sent the following on 12/1/2010 9:16 AM:
>
>>
>
>>
>
>> =========================
>
>> BJ Freeman
>
>> Strategic Power Office with Supplier
>
>> Automation<http://www.businessesnetwork.com/automation/viewforum.php?
>> f
>
>> =52> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems
>
>> Integrator-- Glad to Assist
>
>>
>
>> Chat Y! messenger: bjfr33man
>
>>
>
>>> Thanks
>
>>>
>
>>> I must be missing the spot in web tools it has a complete list of the field names - could you point that out again? For example what is the right field name for the manufacturer / supplier? I did take that out the db....
>
>>>
>
>>> Thanks
>
>>> James
>
>>>
>
>>> -----Original Message-----
>
>>> From: BJ Freeman [mailto:bjfree@free-man.net]
>
>>> Sent: November-30-10 11:42 PM
>
>>> To: user@ofbiz.apache.org
>
>>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>>
>
>>> there should be more to the log that would pinpoint what is going on.
>
>>> I am guessing it is your input to the form, since this has not been changed 09.
>
>>> if you look at the file mentioned you will see
>
>>> <script
>
>>> location="component://webtools/webapp/webtools/WEB-INF/actions/dataf
>>> i
>
>>> l
>
>>> e/viewdatafile.groovy"/>
>
>>> that is where the error referenced.
>
>>>
>
>>> your xml(not xsd) should have field names as you see them in webtools.
>
>>> when you have underscores like manufacturer_party_id then that is what is in the db but ofbiz will not recognize it in Webtools or code.
>
>>>
>
>>>
>
>>>
>
>>>
>
>>> James Lawton sent the following on 11/30/2010 7:34 PM:
>
>>>
>
>>>
>
>>>> I think I am getting closer - thanks for the tips and the patience!
>
>>>>
>
>>>> I have been through all the files and here is what I have gotten
>
>>>> when running my definition file through the data tools
>
>>>>
>
>>>> org.ofbiz.widget.screen.ScreenRenderException: Error rendering
>
>>>> screen
>
>>>> [component://webtools/widget/MiscScreens.xml#viewdatafile]:
>
>>>> groovy.lang.MissingPropertyException: No such property:
>
>>>> definitionUrl for class: viewdatafile (No such property: definitionUrl for class:
>
>>>> viewdatafile)
>
>>>>
>
>>>>
>
>>>> Here is the xsd file I was running:
>
>>>>
>
>>>>
>
>>>> <data-file name="itemdatabasereport.txt" separator-style="delimited" type-code="UTF-8"
>
>>>> delimiter="	" text-delimiter=""
>
>>>> description="this takes the product data from jeffs file"
>
>>>> has-header="true">
>
>>>> <record name="Product">
>
>>>> <field name="manufacturer_party_id" ignored="false"
>>>> type="String"/>
>
>>>> <field name="productId" ignored="false" type="String"/>
>
>>>> <field name="primaryProductCategoryId" expression="true"
>
>>>> default-value="tools" type="String"/>
>
>>>> <field name="productTypeId" expression="true"
>
>>>> default-value="FINISHED_GOOD" type="String"/>
>
>>>> <field name="productName" ignored="false"
>>>> type="String"/>
>
>>>> <field name="description" ignored="false"
>>>> type="String"/>
>
>>>> <field name="quantityIncluded" expression="true" default-value="1"
>
>>>> type="String"/>
>
>>>> <field name="price" ignored="true" type="String"/>
>
>>>> </record>
>
>>>> </data-file>
>
>>>>
>
>>>>
>
>>>> What's wrong with my definition file which is causing the data tool to throw up?
>
>>>>
>
>>>> Thanks
>
>>>> James
>
>>>>
>
>>>>
>
>>>>
>
>>>> -----Original Message-----
>
>>>> From: BJ Freeman [mailto:bjfree@free-man.net]
>
>>>> Sent: November-26-10 1:36 AM
>
>>>> To: user@ofbiz.apache.org
>
>>>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>>>
>
>>>> I am not clear what you are doing.
>
>>>> the example you show is for an entity DataImportProduct which does
>
>>>> not exist in the OOTB ofbiz take a look at
>
>>>>
>
>>>> <https://issues.apache.org/jira/browse/OFBIZ-2534>
>>>> https://issues.apache.org/jira/browse/OFBIZ-2534
>
>>>> for tab examples.
>
>>>> also
>
>>>> <https://issues.apache.org/jira/browse/OFBIZ-2518>
>>>> https://issues.apache.org/jira/browse/OFBIZ-2518
>
>>>>
>
>>>> =========================
>
>>>>
>
>>>> BJ Freeman
>
>>>> Strategic Power Office with Supplier Automation<
>>>> <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>>>> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
>
>>>> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist
>
>>>>
>
>>>> Chat Y! messenger: bjfr33man
>
>>>> James Lawton sent the following on 11/25/2010 6:36 PM:
>
>>>>> I am probably doing something stupid here...I have dug into the
>
>>>>> link that was posted and it looks simple enough...however the code
>
>>>>> snippet is for a fixed width file rather than a comma delimited
>
>>>>> file...and my attempts to create the right schema for importing
>
>>>>> product are failing...If someone has a CSV file snippet to share
>
>>>>> that would be awesome
>
>>>>>
>
>>>>> So heading in a different direction I created the entity data snippet below:
>
>>>>>
>
>>>>> <!-- this file contains some sample products for testing the data
>
>>>>> import routine --> <entity-engine-xml>
>
>>>>>
>
>>>>> <DataImportProduct productId="YG0081ATF"
>>>>> productTypeId="FINISHED_GOOD" description="1/8 x 1-59/64 CARBIDE
>>>>> DREAM DRILL W/O COOLANT HOLES (3XD)" price="22.13"
>>>>> priceCurrencyUomId="USD" supplierPartyId="YG1" purchasePrice=""/>
>
>>>>> <DataImportProduct productId="YG0081BTF"
>
>>>>> productTypeId="FINISHED_GOOD" description="1/8 x 15/64 CARBIDE
>
>>>>> DREAM DRILL W/ COOLANT HOLE" price="105.73" priceCurrencyUomId="USD"
>
>>>>> supplierPartyId="YG1" purchasePrice=""/>
>
>>>>>
>
>>>>>
>
>>>>> </entity-engine-xml>
>
>>>>>
>
>>>>> However every time I have tried to use the XML IMport tool it just
>
>>>>> keeps coming back and saying that the either URL / file or the XML
>
>>>>> are not complete
>
>>>>>
>
>>>>> What am I doing wrong?
>
>>>>>
>
>>>>> -----Original Message-----
>
>>>>> From: BJ Freeman [mailto:bjfree@free-man.net]
>
>>>>> Sent: November-09-10 6:00 PM
>
>>>>> To: user@ofbiz.apache.org
>
>>>>> Subject: [SPAM]Re: How to import a pre existing product database
>
>>>>>
>
>>>>> all web tools imports into memory so is limited to the amount of free memory of the sever.
>
>>>>>
>
>>>>> I imports files that are gigs in size so read in the files. I typical file has 25 columns and takes about 15min.
>
>>>>> There is no need to write code to convert files to xml since the
>
>>>>>
>>>>> <https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+
>>>>> Da>
>>>>> https://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+D
>>>>> a
>
>>>>> t
>
>>>>> a+
>
>>>>> File+Tools
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>> =========================
>
>>>>> BJ Freeman
>
>>>>> Strategic Power Office with Supplier
>
>>>>> Automation<http://www.businessesnetwork.com/automation/viewforum.p
>>>>> hp?f
>
>>>>> =52> Specialtymarket.com< <http://www.specialtymarket.com/> http://www.specialtymarket.com/>
>
>>>>> Systems Integrator-- Glad to Assist
>
>>>>>
>
>>>>> Chat Y! messenger: bjfr33man
>
>>>>> James McGill sent the following on 11/9/2010 12:38 PM:
>
>>>>>
>
>>>>>> On Tue, Nov 9, 2010 at 9:34 AM, James Lawton< <ma...@tofinosoftware.com> jamesl@tofinosoftware.com>wrote:
>
>>>>>>
>
>>>>>>> How would I import a pre existing product database? I currently
>
>>>>>>> have it in a CSV and I am looking for a method of getting that
>
>>>>>>> information into the database so that the imported items will
>
>>>>>>> show up in the catalog
>
>>>>>>>
>
>>>>>>
>
>>>>>>
>
>>>>>> OFBiz ETL is best done as XML seed data.
>
>>>>>> In your scenario I would seriously consider converting the CSV
>
>>>>>> into XML (a job for Perl), and perhaps doing some input
>>>>>> validation
>
>>>>>> as part of that conversion, and then loading the XML as seed data.
>
>>>>>>
>
>>>>>> If you really have a lot of ETL to do (e.g., processing that will
>
>>>>>> take hours or days to run through the service interface) you
>>>>>> might
>
>>>>>> be better off going straight to the database. You lose some of
>
>>>>>> the traceability this way (e.g., logging and event side-effects)
>
>>>>>> so it isn't recommended. (We ran into problems when loading in
>
>>>>>> several years worth of historical data, orders and inventory
>
>>>>>> transactions.)
>
>>>>>>
>
>>>>>> --
>
>>>>>> James McGill
>
>>>>>> Phoenix AZ
>
>>>>>>
>
>>>>>
>
>>>>>
>
>>>>>
>
>>>>
>
>>>>
>
>>>>
>
>>>
>
>>>
>
>>>
>
>>
>
>>
>
>>
>
>