100x Faster Facets and Counts with MongoDB Atlas Search: Public Preview

Today we’ve released one of the most powerful features of Atlas Search in public preview, and ready for your evaluation: lightning fast facets and counts over large data sets.

Faceted search allows users to filter and quickly navigate search results by categories and see the total number of results per category for at-a-glance statistics.

With the new facet operator, facet and count operations are pushed down into Atlas Search’s embedded Lucene index and processed locally – taking advantage of 20+ years of Lucene optimizations – before returning the faceted result set back to the application.

What this means is that now facet-heavy workloads such as ecommerce product catalogs, content libraries, and counts run up to 100x faster.

The power of facets and counts in full-text search

Faceting is a popular search and analytics capability that allows an application to group information into related categories by applying filters to query results. Users can narrow their search results by simply selecting a facet value as a filter criteria. They can intuitively explore complex data sets, providing fast and convenient navigation to quickly drill into the data that is of most interest.

A common use of faceting is navigating product catalogs. With travel starting to reopen, let's take a travel site as an example. By using faceted search, the site can present vacation options by destination region, trip type (i.e. hotel, self-catering, beach, ski, city break), price band, season, and more, enabling users to quickly navigate to the category that is most relevant to them.

Facets also enable fast results counting. Extending our travel site example, business analysts can use facets to quickly compare sales statistics by counting the number of trips sold by region and season.

Prior to the new facet operator, the only way Atlas Search could facet and count data was to retrieve the entire result set back to MongoDB’s internal $facet aggregation pipeline stage. While that was OK for smaller data sets, it became slow when the result set exceeded tens of thousands of documents.

This all changes as now operations are pushed down to Atlas Search’s embedded and optimized Lucene library in a single $search pipeline stage. From our internal testing of a collection with one million documents, the new Atlas Search faceting improves performance by 100x.

How to use faceting in Atlas Search

Our new Atlas Search facets tutorial will help you get started. It describes how to:

  1. Create an index with a facet definition on string, date, and numeric fields in the sample_mflix.movies collection.

  2. Then run an Atlas Search query against those fields for results grouped by values for the string field and by ranges for the date and numeric fields, including the count for each of those groups.

To use Atlas Search facets, you must be running your Atlas cluster on MongoDB 4.4.11 and above or MongoDB 5.0.4 and above. These clusters must be running on the M10 tier or higher. Facets and counts currently work on non-sharded collections. Support for sharded collections is scheduled for next year.

The power of Atlas Search in a unified application data platform in the cloud

MongoDB Atlas Search makes it easy to build fast, relevant full-text search on top of your data in the cloud. A couple of API calls or clicks in the Atlas UI, and you instantly expose your data to sophisticated search experiences that boost engagement and improve satisfaction with your applications. Your data is immediately more discoverable, usable, and valuable.

By embedding the Apache Lucene library directly alongside your database, data is automatically synchronized with the search index; developers get to work with a single API; there is no separate system to run and pay for; and everything is fully-managed for you on any cloud you choose.

Atlas Search provides a fully-integrated full-text search engine, automatically synchronized with your database
Figure 1: Rather than bolting-on a separate search engine to your database, Atlas Search provides a fully integrated platform.

Atlas Search provides the power you get with Lucene — including faceted navigation, autocomplete, fuzzy search, built-in analyzers, highlighting, custom scoring, and synonyms — combining it with the productivity you get fromMongoDB. As a result, developers can ship search applications and new features 30%+ faster.

Next steps

You can try out Atlas Search with the public preview of lightning-fast facets and counts today:

  • If you are new to Atlas Search, simply spin up a cluster (M10 tier or above) and get started with our Atlas Search facets tutorial.

  • If you are already using Atlas Search on M10 tiers and above then update your indexes to use the facet field mapping, and then start querying! Your data remains searchable while it is being re-indexed.

If you want to dig into the use cases you can serve with Atlas Search — along with users who are already taking advantage of it today — download our new Atlas Search whitepaper.

Safe Harbor

The development, release, and timing of any features or functionality described for our products remains at our sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality.

← Previous

Sync or Stink: Why Some Apps Can't Handle Mobile Data

Mobile apps have transformed the way businesses work with data. For example, by automating and digitizing what were previously manual and error-prone tasks, mobile apps have made frontline workers more productive and the operational data they generate more actionable. But that is only true when those workers have modern mobile apps that are fully integrated with back-end systems at the core of the network. Most mobile apps are often slow to load and sync data unreliably, frequently returning stale data or just outright crashing. The reason for this is that data sync is hard. A lot of developer teams take a DIY approach to building sync. But building a sync tool that works in an offline-first environment, like mobile, can take months of complex work and require thousands of lines of code. And when development teams build mobile sync themselves, they often oversimplify the solution — so app data may only sync a few times a day or won’t sync bidirectionally. Building sync the right way — keeping data up-to-date in real-time whenever devices are connected — requires complicated networking and conflict resolution code. MongoDB Realm eliminates this complexity We cover many of the technical difficulties of developing sync tools, including the pitfalls of DIY approaches, in our recent white paper, “ Unlocking Value in Mobile Apps: 3 Best Practices for Mobile Data Sync .” Data sync is absolutely essential for delivering good customer experiences, harnessing the value of mobile data for the business, and ultimately proving that investments in mobile technology are worth it. From getting real-time information to frontline workers to tracking fleets and getting timely inventory updates, mobile data has the potential to deliver true competitive advantage, but only if you’re working with a highly performant mobile platform. When apps work right, users reward you by using the app and, hopefully, leaving positive feedback. Another good sign of a well-built app is that users ask for new features. Development teams that don’t have to worry about complicated conflict resolution code and an overly complex app architecture can focus on quickly developing new features and, in turn, creating better app experiences. MongoDB Realm and Realm Sync include pre-built conflict resolution out of the box. The Realm Mobile Database — used alongside Realm Sync — is object-oriented, so it’s intuitive to mobile developers. Realm enables developers to focus on delivering competitive differentiation instead of worrying about building complicated data sync and conflict resolution tools. This speeds development and enables teams to turn around feature requests faster. Realm Sync works alongside Realm Mobile Database to synchronize data bi-directionally between the Realm Mobile Database on the client and MongoDB Atlas on the back end. Find out more by reading “ Unlocking Value in Mobile Apps: 3 Best Practices for Mobile Data Sync .”

November 3, 2021
Next →

Data and the European Landscape: 3 Trends for 2022

The past two years have brought massive changes for IT leaders: large and complex cloud migrations; unprecedented numbers of people suddenly working, shopping and learning from home; and a burst in demand for digital-first experiences. Like everyone else, we are hoping that 2022 isn’t so disruptive (fingers crossed!), our customer conversations in Europe do lead us to believe the new year will bring new business priorities. We’re already noticing changes in conversations around vendor lock-in, thanks to the Digital Markets Act, a new enthusiasm for combining operational and analytical data to drive new insights faster, and a more strategic embrace of sustainability. Here’s how we see these trends playing out in 2022. Digital markets act draws new attention to cloud vendor lock-in in Europe We’ve heard plenty about the European Commission’s Digital Markets Act , which, in the name of ensuring fair and open digital markets, would place new restrictions on companies that are deemed to be digital “gatekeepers” in the region. That discussion will be nothing compared to the vigorous debate we expect once the EU begins the very tricky political business of determining exactly which companies will fall under the act. If the EU sets the bar for revenues, users, and market size high enough, it’s possible that the regulation will end up affecting only Facebook, Amazon, Google, Apple, and Microsoft. But a European group representing 2,500 CIOs and almost 700 organizations is now pushing to have the regulation encompass more software companies. Their main concern centers around “distorted competition” in cloud infrastructure services and a worry that companies are being locked into one cloud vendor. A trend that will likely increase in 2022 that pushes back on cloud vendor lock-in is embracing multi-cloud strategies. We should expect to see more organisations in the region pursuing multi-cloud environments as a means to improve business continuity and agility whilst being able to access best of breed services from each cloud provider. As we have always said …”it’s fine to date your cloud provider….but don’t ever marry them.” The convergence of operational and analytical data The processing of operational and analytical data is almost always contained in different data systems, each tuned to that use case and managed by separate teams. But because that data lives in separate places, it’s almost impossible for organisations to generate insights and automate actions in real time, against live data. We believe 2022 is the year we’ll see a critical mass of companies in the region make significant progress toward a convergence of their operational and analytical data. We’re already starting to see some of the principles of microservices in operational applications, such as domain ownership, be applied to analytics as well. We’re hearing about this from so many of our customers locally, who are looking at MongoDB as an application data platform that allows them to perform queries across both real-time and historical data, using a unified platform and a single query API. This results in the applications they are building becoming more intelligent and contextual to their users, while avoiding dependencies on centralized analytics teams that otherwise slow down how quickly new, data-driven experiences can be released. Sustainability drives local strategic IT choice Technology always has some environmental cost. Sometimes that’s obvious — such as the energy needs and emissions associated with Bitcoin mining. More often, though, the environmental costs are well hidden. The European Green Deal commits the European Union to reducing emissions by 55% by 2030, with a focus on sustainable industry. With the U.N. Climate Change Conference (COP26) recently completed in Glasgow, and coming off the hottest European summer on record, climate issues have become top of mind. That means our customers are increasingly looking to make their technical operations more sustainable — including in their choice of cloud provider and data centers. According to research from IDC , more than 20% of CxOs say that sustainability is now important in selecting a strategic cloud service provider, and some 29% of CxOs are including sustainability into their RFPs for cloud services. Most interesting, 26% say they are willing to switch to providers with better sustainability credentials. Historically, it’s been difficult to make a switch like that. That’s part of the reason we built MongoDB Atlas — to give our customers the flexibility to run in any region , with any of the three largest cloud providers, and to make it easy to switch between them, and even to run a single database cluster across them. Publicly available information about the footprint of individual regions and even single data centers will make it simpler for companies to make informed decisions. Already, at least one cloud platform has added indicators to regions with the lowest carbon footprint. Source: IDC, European Customers Engage Services Providers at All Stages of Their Cloud Journey, IDC Survey Spotlight, Doc #EUR248484021, Dec 2021

December 21, 2021