Datafiniti Completes The Last Phase of Elasticsearch Migration with Overhaul of Property API

Datafiniti
Knowledge from Data: The Datafiniti Blog
3 min readNov 9, 2017

--

Datafiniti is back to drop its final Elasticsearch update.

Back in February and June, the team released its revamped database and enhanced Product and Business API functionality following a complete backend migration from a Cassandra-based data warehouse to Elasticsearch. After a two month process, today, Datafiniti announces it now offers the same features and functionality for its Property API. The complete migration will enable the team to utilize a database that is line with both company-wide needs and skillsets. — Tweet about this

“This concludes our migration to ElasticSearch, ending a year and a half project,” said Phil Coleman, chief technology officer. “With ElasticSearch serving as a strong foundation, we can now turn our attention to exciting new ways to improve the scope and quality of our data.”

Similar to the Product and Business API, Property API improvements will provide customers with the following:

  • Faster updates from web data extraction
  • Expanded query capabilities on new fields
  • Accelerated delivery of downloads and results files

“Customers will enjoy a user experience more attuned with today’s technical expectations on consistency, speed, and flexibility,” said Mark Hess, senior data engineer.

Already a Datafiniti customer? Here is what these product changes mean for you.

More data from web data extraction. Customers will have consistent access to well-integrated, varied, and valid property data. Data collected from web data extraction will populate our database more efficiently. With Elasticsearch, the results will include daily accelerated updates and the addition of millions of records.

Expanded query capabilities on new fields. Customers will be able to query every field included in our schema, in addition to new and unique sub-objects. These fields will allow customers to craft ultra-specific requests to our database and to track property trends. Limitations with our previous back-end prevented users from querying data contained within “multi-valued” fields such as prices, reviews, and descriptions.

Accelerated delivery of downloads and results files. Customers will see a quick turnaround on data results as API response times are reduced from seconds to milliseconds and download times from hours to minutes. In addition to acquiring their tailor-made data, customers can also download our entire business database, in less than three hours. With Elasticsearch, customers can say goodbye to database access and reliability concerns.

“Property data is varied and requires extensive domain knowledge,” said Will Hudgins, data engineer. “Our new schema is built upon over two years of experience and learning what did and didn’t work for our customers.”

You can connect with us and learn more about our business, people, product, and property APIs and datasets by selecting one of the options below.

Our team looks forward to seeing how the outcomes of this release will empower your team to streamline its workflow and maximize productivity as we collaborate on fulfilling your data needs.

For the full list of other improvements and bug fixes, check out our release notes. If you have any questions, you can contact your customer success representative or email our team at support@datafiniti.co.

Written by Nicholle Shaver, content marketer.

--

--

We provide instant access to business, people, product, and property data sourced from thousands of websites.