Back to Blog

[RESOLVED] PartsTech Support: Temporary System Outage Nov. 19 2020

On Thursday, November 19th we experienced a system issue related to our search index. The issue has been resolved as of 6 PM EST Thursday evening.

This issue impacted vehicle-related searches across the majority of vendors. Please note Advance Auto Parts and Autozone were not impacted by this issue. Also, please note that users could continue to search and order parts by part number across all vendors - only vehicle-related searches were impacted by this issue. PartsTech Tires was unaffected by this issue and remained fully operational.

Progress Updates:

Update 10:36 AM EST: Our most frequently searched brands are back online for all vendors. Users will see results for the most common brands and part types.

Update 12:16 PM EST: NAPA and WorldPac are back online.

Update 3:06 PM EST: 75% of vehicle-related searches are fully operational including the most popular brands & vendors. The issue should be completely resolved by 6 PM EST today.

Update 6 PM EST: The issue has been completely resolved for all brands across all vendors.

Full technical description from our CTO:

"On Thursday, November 19, at 3:40 AM, while running an automatic backup of our search index, the scripts hadn't finished the operation successfully. As a result, it corrupted metadata of the search index and invalidated backups. It didn't affect our parts or application data, which resides in a completely different part of the infrastructure, however, it hampered our ability to run application searches. Our engineering team quickly configured a new search cluster using pre-built schema and started a process of full indexing from the parts and application databases. Due to the volume of our catalog, the process took 8 hours to complete. However, we prioritized the most selling brands and the most purchased vehicle data first, giving our clients the ability to work with the catalog before the process was completed.

We understand the importance of our search capability to our clients and are working hard right now to analyze the outage and prevent this from happening in the future. We will ensure that the backup process is running smoothly and add additional infrastructure to our Beta server to ensure proper testing of the scripts and the operation workflows. We will also adjust our Production cluster and add more redundancy to the search index and related infrastructure."

Olexy Sadovy CTO, PartsTech Inc.