Import ICEcat product information

Requirements

• CSVI 4+
• VirtueMart 2.0.2+
• ICEcat account

Setting up ICEcat

To be able to use CSVI with ICEcat, a few things need to be setup first. These things are:

1. ICEcat account
2. Index files need to be loaded
   
   1. Files index
   2. Manufacturer index

ICEcat account

An ICEcat account can be requested at the ICEcat website.

Index files

There are two index files that need to be loaded:

1. Files index
   The files index is a list of URLs that are available on the ICEcat server. This file must be loaded into the
   database as CSVI VirtueMart will check this list if product information is available. The default file index is
   located at http://data.icecat.biz/export/freexml.int/INT/files.index.csv
2. Manufacturer index
   The manufacturer index is a list of manufacturers that are available on the ICEcat server. This file must be
   loaded into the database as CSVI VirtueMart will check this list if product information is available. The
   default file index is located at http://data.icecat.biz/export/freexml.int/INT/supplier_mapping.xml

The index files are loaded via the Maintenance menu:

On loading the index files, CSVI follows this process:

1. Check if the file icecat_index exists in the folder set by ICEcat indexes path
2. If 1 does not exist, check if the file icecat_index.gzip exists in the folder set by ICEcat indexes path
3. If 2 does not exist, check if the file icecat_index.zip exists in the folder set by ICEcat indexes path
4. Check if the file icecat_supplier exists in the folder set by ICEcat indexes path
5. If 4 does not exist, check if the file icecat_supplier.gzip exists in the folder set by ICEcat indexes path
6. If 5 does not exist, check if the file icecat_supplier.zip exists in the folder set by ICEcat indexes path

If any of the above files are found, CSVI will use these local files to import the indexes. If none of the files are found, CSVI will try to download the files from the ICEcat server.

In case you have troubles loading the files index make sure the ICEcat indexes folder does not have any files in it.

Use gzip

This option is useful to download the large files index file as the size of this file is more than 100MB unpacked. The download is usually no problem but after the file is downloaded CSVI needs to unpack the zip file and can easily run out of memory. There are 3 solutions for this:

1. Increase the available memory
2. Download the unzipped file
3. Unzip the file on your computer and upload it to the server

Load the files index in 1 step

The files index is a big file with over 500.000 entries. By default CSVI will load the files index in 1 step because this is the fastest and most efficient way of loading the files index.  
 
To be able to load the file in 1 step the INFILE command needs to be enabled on the server. When the INFILE command is not enabled the error: Error occurred: The used command is not allowed with this MySQL version appears. To be able to load the index files, the MySQL needs to be configured to use the
INFILE command. This command can be enabled in the MySQL configuration file.

Add this line:

local-infile=1
to the /etc/my.cnf in the [mysql] section.
 
This is most likely done by the webhost.

Load the files index in multiple steps

In case it is not possible to load the files index file in 1 step, it is possible to load it in multiple steps.
 
 
To load the file in multiple steps set the following options:

• Load type: Single
• Number of lines per import: 1000
  This is the number of lines that is imported on each step.
• Wait time between imports: 5
  This time is when the import pauses to reduce the load on the server

When performing the ICEcat index loading using cron, the 1 step import is used.

Manufacturer Part Number

ICEcat identifies products based on the MPN (Manufacturer Part Number) and manufacturer name. If your product SKU is not the same as the MPN you must add a field to the VirtueMart product table to store the MPN.

On import you must also set in your template settings to use the MPN column instead of the product SKU as shown below. In this example the database column is called product_mpn.

What happens now on import is that not the column product_sku is used as unique identifier but the column product_mpn.
 
In case the product_sku is the only column and contains the MPN value, the custom column name can be set to product_sku.

Enabling ICEcat Data

To tell CSVI that not the data in the import file needs to be used but to use ICEcat data instead, enable the “Use ICEcat data” option as shown above.

Similar SKU search

The similar SKU search will search ICEcat for SKUs that are similar to the one being imported. This helps to find products that have an added identifier. For example the SKU A123 is imported but in ICEcat it is known as A123AA, CSVI will take the information from the A123AA ICEcat file.

Preparing the Import

CSVI will update the following fields with ICEcat data if found in the import file:

• category_path
• manufacturer_name
• product_sku
• product_name
• product_desc
• product_s_desc
• product_available_date
• file_url
• file_url_thumb
• features
• related_products

When importing ICEcat data only include the above fields you want updated with ICEcat information.

Example:

Import ICEcat Data

To import the ICEcat data the regular import process is followed. Use the Product Import to load the ICEcat data.

Supported Languages

INT - International standardized version of a data-sheet
EN – Standard or UK English
US – US English
NL – Dutch
FR – French
DE – German
IT – Italian
ES – Spanish
DK – Danish
RU – Russian
PT – Portuguese
ZH – Chinese (simplified)
SE – Swedish
PL – Polish
CZ – Czech
HU – Hungarian
FI – Finnish
NO – Norwegian
TR – Turkish
BG – Bulgarian
KA – Georgian
RO – Romanian
SR – Serbian
JA – Japanese
UK – Ukrainian
CA – Catalan
HR – Croatian