Searchspring Live Indexing and Pricing

Released In: 10.04.01

Overview

When using third-party tools to power search pages (and categories) it is critical that the product data they have in their system is as up to date as possible to prevent the display of stale data.

Searchspring provides a “Live Indexing API” which allows Miva to push up product changes in real time as they happen. That means Searchspring does not have to wait for the next interval of a bulk feed import to get the product changes. This is extremely helpful when new products are being added frequently, prices are being changed or inventory changes are causing products to go into or out of stock.

On this Page

• Setup and Installation
• Searchspring Queue
• Template Functions
• Process Queue - Send Product Updates to Searchspring
• Visual Diagram of Workflow
• Scheduled Task Timing
• Bulk Indexing Status Check
• JSON Functions
• XML Provisioning
• Using Product Data Fields and Fees with Searchspring

Setup and Installation

To install the module, click the Settings header on the left side of the page, then click Modules and search for "Searchspring."

When installation is complete, click the User Interface header on the left side of the page, then click Global Settings . Next, click More and then click Searchspring Live Indexing and Pricing .

• Site ID : The identifier for your Searchspring site. It can be found on the My Account page of the Searchspring Management Console.
• Secret Key : The secret key for your live indexing API. Do not share this. Please contact support@searchspring.net for your secret key.
• Automatic Deletion - Feeds : Comma Separated List of the Feeds Ids you want to enable automatic deletion for. Automatic deletion means that when a product is deleted from Miva it will also be deleted in Searchspring
• Automatic Deletion - ID Field : This is the unique identifier Miva will pass in the records object in the delete API call. This must match your Searchspring ID Field.
  • id - Miva Product ID
  • code - Miva Product Code
  • sku - Miva Product SKU

Searchspring Queue

The module maintains its own queue of products that need to be updated in Searchspring. A product could need to be pushed back to Searchspring for any number of reasons including:

• The price changed or it went on/off sale
• Inventory changes (In stock/Out of Stock)
• Active/Inactive
• Category Assignments changed
• Custom Field Values Changed

The only way to add/update/delete a product from the Searchspring Queue is via the template-level functions defined below.

The main benefit of queueing products is it allows Miva to group the request together and have greater control over the 2.5Mb data limit or a Max of 500 records.

Force Delete Products From Queue

Under the module settings, there is a button to force clear any products from the Searchspring queue. Its located under the More menu in the module's admin configuration screen.

Template Functions

Upon installation of the module, a new item is automatically created named “searchspringlive.” This item needs to be assigned to any page template you wish to use the following functions:

Parameter Definitions

Flags : may be an empty string or "replace." If "replace" is specified, any existing entry with status "Q" for the product_id/feed_id combination of the same type, is replaced by the new one. Otherwise, any existing entries are retained and will be processed in the order in which they were queued.

For example, let’s say there were multiple updates to the same product with each update adding the product to the Searchspring queue. If you place a flag value of “replace,” only the latest record in the queue will be processed. If the flag parameter is empty, each record will be sent to Searchspring, processed in the order in which they were queued, creating duplicate data that Searchspring must consume.

product_id : The id of the product to add to the queue.

feed_id : The Searchspring feed_id. You must email support@searchspring.com to get this value for your account.

data : This is a JSON object of the full product record to be updated. You cannot update partial records, for example, just sending the price. It must be the full data set for the product you wish to send to Searchspring. These fields could be different on a per customer basis as custom fields you send to Searchspring are unique to you.

Delete Example

When using the delete function to add a product to be deleted to the queue, the data value needs to be a JSON encoded array of product ids to delete from Searchspring. When using this function it is up to the caller to ensure the data is properly formatted. Whatever value you pass into for the data parameter will be wrapped in the records array:

"records":[' $ l.request_data $ ']

<mvt:assign name="g.Data" value="'\"1\"'" />
<mvt:item name="searchspringlive" param="Queue_Delete( 1, 1111, g.Data )" />

A common place to utilize these template functions to add/update/delete products from Searchspring is in the Product Data Fields and Feeds module. It allows you to hook into the new Product Notification APIs and conditionally send the product to Searchspring if warranted.

Process Queue - Send Product Updates to Searchspring

The Searchspring queue is processed via a Scheduled Task. This scheduled task needs to be added after the module has been installed.

To set this up, click the Settings header on the left side of the page, then click Store Settings. Finally, click the Scheduled Tasks tab. Once there, click Create New Scheduled Task.

In the Operation dropdown, choose Searchspring Live Indexing and Pricing: Process Searchspring Live Update Queue.

Visual Diagram of Workflow

Scheduled Task Timing

The timing of how often this scheduled task runs can be configured from this screen. However, in most cases, since you want the data updated in Searchspring as soon as possible after the update occurs, it will be better to leverage triggers to tell Miva to processes the queue.

Bulk Indexing Status Check

Just before the Scheduled Task is set to run, Miva will make a call to Searchspring to see if there is currently a Bulk Index in process. If yes, this means that Searchspring is currently processing a bulk data feed and the live indexing updates will wait until this process is complete. An informational message will be logged in Miva if the task is prevented from running.

JSON Functions

The module provides a runtime JSON API function that is used on the website to get real-time, customer-level pricing and inventory levels for an array of products. This is useful when needing to update the pricing returned from the Searchspring API for specific customers.

This function will allow you to pass on an array of product codes or Ids and get back the current price (including any discounts) and inventory counts. This function is intended to be called in front end JavaScript for when a third-party search provider may not have the most up to date pricing or discount data for products.

XML Provisioning

Below are the XML Provisioning Tags that are available: 

<Module code="searchspringlive" feature="scheduledtask">
    <Settings_Update>
        <SiteID>...</SiteID>
        <Secret>...</Secret>
        <AutomaticDeletionFeeds>123,456,789</AutomaticDeletionFeeds>
        <AutomaticDeletionField>ID|Code|SKU</AutomaticDeletionField>
    </Settings_Update>
</Module>
 

Using Product Data Fields and Fees with Searchspring

One of the primary ways to add a product to the Searchspring Queue to be sent to Searchspring's Live Indexing is to use the Product Data Fields and Feeds module. It allows you to use a page template to generate and store product data that can eventually be output on a feed. The benefit of this module is that as product data changes it regenerates the output immediately rather than in bulk with traditional feeds. The benefit is that very large product data feeds can output in seconds instead of hours.

Searchspring provides a bulk feed format using line delimited JSON. Using the product data Field and Feeds module, you can use a single data field page template to both generate the JSON for the bulk feed as well as add the product to the live indexing queue. Below is a simplified example, which would need to be modified to your specific data set to be valid.

<mvt:miva output="off" />
 
<mvt:comment>
|
| Output from this page will be stored as the generated product data.
|
| Empty output will delete an existing record for a product.
|
</mvt:comment>
 
 
<mvt:capture variable="g.ssdata">
{
    "product_id": "&mvtj:product:id;",
    "name": "&mvtj:product:name;",
    "price": &mvtj:product:price;,
    "categories": [
        "Clothing",
        "Shirts"
    ],
    "image": "&mvtj:product:imagetypes:main;",
    "thumbnail": "&mvtj:product:imagetypes:main;",
    "url": "&mvtj:product:url;",
    "in_stock": true,
    "rating": 4.5,
    "brand": "Nike",
    "color": "Blue",
    "popularity": 9
}
</mvt:capture>
 
 
<mvt:comment>
| Queue For Searchspring
</mvt:comment>
 
<mvt:item name="searchspringlive" param="Queue_Upsert( 'replace', l.settings:product:id, '12345', g.ssdata )" />
 
 
<mvt:comment>
| Output data to save Generated Data to Custom Field
</mvt:comment>
 
&mvtj:global:ssdata;

Scheduled Task Handoff

The Searchspring Live Indexing Scheduled Task to process the feed can also be setup run in conjunction with a Product Data Field Scheduled task. For example, if you update prices for 10,000 products either via an API integration to your ERP, or just a simple CSV import, this will trigger the Product Change notification and add those 10,000 products to the Queue to be processed by the Product Data Fields and Feeds module.

However, these changes also need to be pushed to Searchspring. By default all 10,000 products would get added to and processed for Product Data Fields then a seperate task would process the Searchspring Queue. This means there could be a delay from when the change was made in Miva and when we tell Searchspring about it.

In the Scheduled Task for Product Data Field and Feeds, there are settings to limit how many records get processed and then hand over execution to the scheduled task for Searchspring to process its queue. In the example below, only 1,000 records will get processed, then the trigger on completion setting will start the Searchspring scheduled task pushing up those 1,000 records to Searchspring. This gives you control to fine tune how quickly the product changes get pushed to Searchspring.