Leveraging Search Stages with Filters, Sorting, and Search Terms

In my last engineering blog, we explored Octopart’s search technology — I explained how the Octopart search engine matches part number searches to the parts stored in our database. Continuing in the spirit that it’s easier to use a tool when you understand how it works, this blog will discuss the ways you can control or change search results on Octopart, and how those functions work.

The search engine has a big task. There are over 40 million parts on Octopart but the first page of results shows a maximum of 10. Surfacing the best ten parts out of over 40 million is a big challenge! It happens in a couple of steps.

The first stage of processing a search request is determining the result set: of the over 40 million parts in the collection, which ones “match”? This step is a binary choice: each part is either matches or does not. The set of matching parts is called the result set or the “hits.” In the above example there are 3263 hits.

Next, the search engine orders the result set. Of those 3263 hits, which comes first, second, third, and so on?

Finally the the result set is split into pages (on Octopart, there's 10 parts per page). So the parts you see on the first page are the first 10 hits in the ordered list of 3263 hits, which is the matching subset out of the total collection of over 40 million parts.

When you’re doing a search on Octopart, there are several ways that you can control these steps: what search terms you type into the search bar, what filters you add, and what sorting preferences you choose. We all interact with search engines many times per day and use functions like this to help get the results we need. But often we use these functions without considering them individually. This makes it easy to overlook the details of what they’re actually doing and how you can use them best. Let’s look at each more closely.

The most basic way to impact your search results is by adding filters. These are visually represented in the left sidebar of the search results. A filter selects a set of matching parts, without affecting the ordering of those parts. A filter is binary: either a part matches a filter and is included in the result set or it doesn’t match and is not included.

Filters are combined according to boolean logic.

Selecting the “LEDs” category applies a category filter. Out of the total part collection, there are 56,986 parts with the category “LED”. The filter does not impact the ordering of the results. In this example the default Octopart sort order is used, which is designed to bring available, quality parts to the top.

Adding the Manufacturer filter makes the result set smaller: 2,022 hits. These parts are in the LED category AND manufactured by Lite-On. Different filters are combined with a boolean AND. We can think about this with set theory: the category filter defines Set A, the manufacturer filter defines Set B, when we apply both filters, the result set is the intersection of Set A and Set B.

If we select more than one manufacturer we get more results (5,003, still less than the total number of parts in LEDs).

The results are parts matching:

(category LEDs) AND (manufacturer Avago OR Lite-On)

Multiple values for a single filter are combined with a boolean OR. Again, changing these filters only changes which parts are hits, not the order of the hits.

The next thing we can do is to directly impact the order of the results by choosing a Sort preference.

Instead of the default Sort by Relevance, we can choose to order the result set from lowest to highest price. This does not change the total number of hits, which is still 5,003, or which parts hit, but by ordering them differently we see new parts on the first page.

If we choose Matrix view, we can sort by any part attribute.

For example, sort by Viewing Angle descending to see the parts with the greatest viewing angle.

The final function available is the most complex, and was maybe the first thing you thought of: the search terms! Those words you type in the box.

Search terms are particularly powerful because they impact both part matching and ordering. Given a set of search terms like red SMD LED the search engine chooses the set of parts that “match” the search terms (that's a binary match/no-match choice like a filter). But it also measures how well each part matches the query. This impacts the order of the results. On Octopart that how well calculation encompasses the confidence that each part is a red smd led, as well as supply chain and other part quality data.  

So what’s required for a part to “match” a set of search terms?

Octopart shows parts that contain all the words in the search box. Each part is represented as a document with a set of fields, like manufacturer name, part number (MPN), descriptions, category, and more. A search query like red SMD LED is split on whitespace into distinct words. A part matches when all of the words appears in at least one field. Of course, the concept of a “word” is a little more involved because it includes MPNs and partial MPNs.

We’ve found that this strategy is the best way to provide search results that meet users’ expectations. You get all the parts that reasonably match your query and you don’t get back parts that only correspond with some of your terms. 

Search engines are powerful tools to help humans interact with large volumes of information. Octopart part database is a tremendous resource, but too large to process without advanced tools. By developing an accurate mental model of the search engine machine and the levers available to control it, it becomes easier to manipulate all the information available to us — on Octopart and in the world.