MongoDB Developer
Coding with MongoDB - news for developers, tips and deep dives
MongoDB Hackathon is Here
December 13, 2021
Developer
Data and the European Landscape: 3 Trends for 2022
The past two years have brought massive changes for IT leaders: large and complex cloud migrations; unprecedented numbers of people suddenly working, shopping and learning from home; and a burst in demand for digital-first experiences. Like everyone else, we are hoping that 2022 isn’t so disruptive (fingers crossed!), our customer conversations in Europe do lead us to believe the new year will bring new business priorities. We’re already noticing changes in conversations around vendor lock-in, thanks to the Digital Markets Act, a new enthusiasm for combining operational and analytical data to drive new insights faster, and a more strategic embrace of sustainability. Here’s how we see these trends playing out in 2022. Digital markets act draws new attention to cloud vendor lock-in in Europe We’ve heard plenty about the European Commission’s Digital Markets Act , which, in the name of ensuring fair and open digital markets, would place new restrictions on companies that are deemed to be digital “gatekeepers” in the region. That discussion will be nothing compared to the vigorous debate we expect once the EU begins the very tricky political business of determining exactly which companies will fall under the act. If the EU sets the bar for revenues, users, and market size high enough, it’s possible that the regulation will end up affecting only Facebook, Amazon, Google, Apple, and Microsoft. But a European group representing 2,500 CIOs and almost 700 organizations is now pushing to have the regulation encompass more software companies. Their main concern centers around “distorted competition” in cloud infrastructure services and a worry that companies are being locked into one cloud vendor. A trend that will likely increase in 2022 that pushes back on cloud vendor lock-in is embracing multi-cloud strategies. We should expect to see more organisations in the region pursuing multi-cloud environments as a means to improve business continuity and agility whilst being able to access best of breed services from each cloud provider. As we have always said …”it’s fine to date your cloud provider….but don’t ever marry them.” The convergence of operational and analytical data The processing of operational and analytical data is almost always contained in different data systems, each tuned to that use case and managed by separate teams. But because that data lives in separate places, it’s almost impossible for organisations to generate insights and automate actions in real time, against live data. We believe 2022 is the year we’ll see a critical mass of companies in the region make significant progress toward a convergence of their operational and analytical data. We’re already starting to see some of the principles of microservices in operational applications, such as domain ownership, be applied to analytics as well. We’re hearing about this from so many of our customers locally, who are looking at MongoDB as an application data platform that allows them to perform queries across both real-time and historical data, using a unified platform and a single query API. This results in the applications they are building becoming more intelligent and contextual to their users, while avoiding dependencies on centralized analytics teams that otherwise slow down how quickly new, data-driven experiences can be released. Sustainability drives local strategic IT choice Technology always has some environmental cost. Sometimes that’s obvious — such as the energy needs and emissions associated with Bitcoin mining. More often, though, the environmental costs are well hidden. The European Green Deal commits the European Union to reducing emissions by 55% by 2030, with a focus on sustainable industry. With the U.N. Climate Change Conference (COP26) recently completed in Glasgow, and coming off the hottest European summer on record, climate issues have become top of mind. That means our customers are increasingly looking to make their technical operations more sustainable — including in their choice of cloud provider and data centers. According to research from IDC , more than 20% of CxOs say that sustainability is now important in selecting a strategic cloud service provider, and some 29% of CxOs are including sustainability into their RFPs for cloud services. Most interesting, 26% say they are willing to switch to providers with better sustainability credentials. Historically, it’s been difficult to make a switch like that. That’s part of the reason we built MongoDB Atlas — to give our customers the flexibility to run in any region , with any of the three largest cloud providers, and to make it easy to switch between them, and even to run a single database cluster across them. Publicly available information about the footprint of individual regions and even single data centers will make it simpler for companies to make informed decisions. Already, at least one cloud platform has added indicators to regions with the lowest carbon footprint. Source: IDC, European Customers Engage Services Providers at All Stages of Their Cloud Journey, IDC Survey Spotlight, Doc #EUR248484021, Dec 2021
Joyce, a Decentralized Approach to Foster Business Agility
Despite all of the tools and methodologies that have arisen in the last few years, many companies, particularly those that have been in the market for decades, struggle when it comes to leveraging their operational data to build new digital products and services. According to research and surveys conducted by McKinsey over the last few years, the success rate of digital transformations is consistently low, with less than 30% succeeding at improving their company’s performance. There are a lot of reasons for this, but most of them can be summarized in a sentence: A digital transformation is primarily an organizational and cultural change and then a technological shift. The question is not if digital transformation is a good thing nor is it if moving to the cloud is a good choice. Companies need (badly, in some cases) a digital transformation and yes, the pros of moving to the cloud usually overcome the cons. So, let’s try to dig deeper and analyze three of the main problems companies face when they go on this journey Digital products development Products by nature are customer-driven but companies run their businesses on multiple back-end systems that are instead purpose-driven. Unless you run a very small business, different people with different objectives have ownership of such products and systems. Given this context, what happens when a company wants to launch a new digital product at speed? The back-end systems (CRMs, E-commerce, ERP, etc.) hold the data they need to bring to the customer. Some systems are SaaS, some are legacy, and perhaps others are custom applications created by the company that disrupted the market with innovative solutions back in the days, the perfect recipe for integration hell. The product manager needs to coordinate and negotiate multiple change requests with the system’s owners whilst trying to convince them to add their needs in the backlog to meet the deadline. And things get even worse, as the new product relies on the computational power of the source systems, and if those systems cannot handle the additional traffic, both the product and the core services will be affected. Third-party integration “Everybody wants the change, (almost) nobody wants to change.” In this ever-growing digital world, partnering with third parties (whether they are clients or service providers) is crucial, but everyone who has tried to do so knows how challenging this is: non-standard interfaces, CSV files over FTP with fancy update rules, security issues… The list of unwanted things can grow indefinitely. SaaS everywhere The Software-as-a-Service model is extremely popular and getting the service you want without worrying about the underlying infrastructure gives freedom and speed of adoption, but what happens when a big company relies on multiple SaaS products to run their business? Sooner or later, they experience loss of control and higher costs in keeping a consistent view of the big picture. They need to deal with SaaS internal representations of their own data, multiple views of the same domain concept, unplanned expenses to export, and interpret and integrate the data from different sources with different formats. Putting it all together All the issues above fall into a well-known category of information technology. They are integration problems, and over the years, a lot of vendors promised a definitive solution. Now, you can consider low-code/no-code platforms with hundreds of ready-made connectors and modern graphical interfaces. Problem solved, right? Well, not really. Low-code integration platforms simplify implementation. They are really good at it, but doing so oversimplifies the real challenge: creating and maintaining a consistent set of APIs shaped around the business value over time, and preventing the interfaces from leaking internal complexities to the rest of the company, something that has to be defined and maintained through architectural choices and proper skills (completely hidden behind the selling points of such platforms). There are two different ways to solve integration problems: Centralized using adapters. In this case, the logic is pushed to the central orchestration component, with integration managed through a set of adapters. This is the rather old school SOA approach, the one that the majority of market integration platforms are built on. Decentralized, pushing the logic to the edges, giving autonomous teams the freedom to define both the boundaries and the APIs that a domain must expose to deliver business value. This is a more modern approach that has arisen recently alongside the rise of microservices and, in the analytical world, with the concept of data mesh. The former gives speed at the starting point and the illusion of reducing the number of choices and skills to manage the problems, but in the long run, inevitably, this begins to accumulate technical debt. Due to the lack of necessary degrees of freedom, you lose the ability to evolve the integration points over time, the same thing that caused the transition from SOA to microservices architectures. The latter needs the relevant skills, vision, and ability to execute but gives immediate results and allows you to flexibly manage the evolution of the enterprise architecture over time. Old problems, new solutions At Sourcesense in the last 20 years, we have partnered on hundreds of projects to bring agility, speed, and new open-source technology to our customers. Many times through the years, we were faced with the integration challenges above, and yes, we tried to solve them with the technology available at the time, so we have built some integration solutions on SOA (when they were the best of breed) and interacted with many of the integration platforms on the market. Then, we struggled with the issues and limitations of the integration landscape and have listened to our customers’ needs and where expectations have fallen short. The rise of agile methodologies, cloud computing, new techniques, technologies, and architectural styles has given an unprecedented boost to software evolution and the ability to support business needs, so we embraced the new wave and now have growing experience in solving problems with these tools. Along the way, we’ve seen a recurring pattern when we encountered integration problems, the effectiveness of data hubs as components of the enterprise architectures to solve these challenges, so we built one of our own: Joyce. Data hubs This is a relatively new term and refers to software platforms that collect data from different sources with the main purpose of distribution and sharing. Since this definition is broad and vague, let’s add some other key elements that matter and help define the contours of our implementation. Collecting data from different sources can bring three major benefits: Computational decoupling from the sources. Pulling (or pushing) the data out of the originating systems means that client applications and services interact with the hub and not directly with the sources, preventing them from being slowed down by additional traffic. Catalog and discoverability. If data is collected correctly, this leads to the creation of a catalog, allowing people inside the organization to search, discover, and use the data inside the hub. Security. The main purpose of the hubs is distribution and sharing. This leads immediately to focus on access control and security hardening. A single access point simplifies the overall security around the data because it significantly reduces the number of systems the clients have to interact with to gather the data they need. Joyce, how it works The cornerstone concept of Joyce is the schema. It allows you to shape the ingested data and how this data will be made available to client services. Using the same declarative approach made popular by Kubernetes, the schemas describe the expected result and the platform performs the actions to make it happen. Schemas are standard JSON schema files stored and classified in a catalog. Their definition falls into three categories: Input – how to gather and shape the source data. We leverage the Kafka Connect framework to provide ready-made connectors for a wide variety of sources. The ingested data can be filtered, formatted, and enriched with transformation handlers (domain-specific extensions of JSON schema). Model – allows you to create new aggregates from the data stored in the platform. This feature gives the freedom to model the data the way needed by client services. Export – bulk data export capability. Exported data can be any query run against the existing data with an optional temporal filter. Input and model data is made available to all the client services with the proper authorization grants through auto-generated REST and GraphQL APIs. It is also possible to subscribe to a dedicated topic if an event-driven approach is more suitable for the use-case. MongoDB: the key for a flexible model and performance at scale We heavily rely on MongoDB. Thanks to its flexibility, we can easily map any data structure the user defines to collect the data. Half of the schema definition is basically the definition of a MongoDB schema. (We also auto-generate one schema per collection to guarantee data integrity.) Joyce runs in a Kubernetes cluster and all its services are inherently stateless to exploit the full potential of horizontal scaling. The architecture is based on the CQRS pattern. This means that writes and reads are completely decoupled and can scale independently to meet the unique needs of the production environment. MongoDB is also the backing database of the API layer so we can keep the promise of low latency, high throughput, and continuous availability along all the components of the stack. The platform is available as a fully managed PaaS on the three major cloud providers (AWS, Azure, GCP) but if needed, it can be installed on an existing infrastructure (in cloud and on prem). Final considerations There are many challenges leaders must face for a successful digital transformation. They need to guide their organizations along a process that involves changes on many levels. The exponential growth of technological solutions in the last few years adds more complexity and confusion. The evolution of organizational models and methodologies point in the direction of shared responsibility, people empowerment, and autonomous teams with a light and effective central governance. The same evolution also permeates the novel approaches to enterprise architectures like the data mesh. Unfortunately, there’s no silver bullet, just the right choices for the given context. Despite all the marketing and hype around this or that one solution to all of your digital transformation needs, a long term successful shift needs guidance, competence and empowerment. We’ve built Joyce with the aim of reducing the burden of repetitive tasks and boilerplate code to get the results faster and catch the low hanging fruits without trying to replace the necessary architectural thinking to properly define the current state and the evolution of the enterprise architectures of our customers. If you’re struggling with the problems enlisted at the beginning of this article you should give Joyce a try. Learn more about Joyce
FHIR Technology is Driving Healthcare's Digital Revolution
Technology supporting healthcare’s digital transformation is so pervasive that the question isn’t what technology to choose, but rather, what problems need to be solved. Advancing technology and access to secure and real-time data analytics will vastly improve patients’ health and happiness, and growing interoperability standards are pushing organizations forward in their digital transformations. Together with the Healthcare Information and Management Systems Society (HIMSS) and leading healthcare insurance provider Humana , MongoDB recently released a three-part podcast series chronicling the ways Fast Healthcare Interoperability Resources (FHIR), AI, and the cloud are reshaping healthcare for the better. Here’s a quick roundup of our discussions. Data is the future of healthcare . Whether providers are driving patient engagement through wearable devices, wellness programs or connected care, data will take healthcare to the next digital frontier. We’ll see these advancements through AI, FHIR, and the cloud. FHIR is revolutionizing healthcare technology . Not only is FHIR implementation a requirement, it’s also a crossroads for data architects. Choosing the right approach has deep implications for healthcare IT. The operational data layer (ODL) approach to interoperability makes the impossible possible . Through Humana’s digital transformation journey, it became clear that meaningful progress isn’t possible using core legacy database systems. AI, FHIR, and the cloud: Why data is the future of healthcare In this episode , we dive into what a digital transformation would look like for the healthcare industry, and what are some of the biggest technology challenges facing healthcare today. A digitally transformed healthcare industry will weave real-time data analytics with more personalized care. Patients today want a more modern healthcare experience that includes telemedicine, digital forms and touchless mobile check ins. The end goal is simple: maximize the human experience while advancing away from legacy technology systems that slow down both healthcare practitioners and patients. When it comes to today’s biggest healthcare challenges, the cloud stands out as a key driver of promise and peril. The promise is that we can build applications, go to market and reach patients through wellness programs more quickly. The peril lies in the infrastructure, which is unknown to many healthcare organizations. This presents a unique challenge for the architects and certainly the developers at organizations with older legacy systems. The challenge here is avoiding a simple left hand shift or cloud for the sake of cloud, and moving from simple modernization to actual transformation. Listen below to hear the entire conversation Your browser does not support the audio element. Bring the FHIR inside for digital transformation In episode 2 , HIMSS and MongoDB take a closer look at why FHIR is a change agent in healthcare technology, and how healthcare organizations globally are using the new data standard to jump start legacy modernization and digital transformation. What is FHIR? The FHIR standard is a common set of schema definitions and APIs that helps providers and patients manage and exchange healthcare data. Using FHIR, records provided by healthcare organizations are standardized into a common data model over rest-based APIs. It makes the data that healthcare providers and payers use easier to exchange. Growing regulatory pressure has accelerated U.S. FHIR adoption among healthcare organizations and technology vendors.The Centers for Medicare and Medicaid Services (CMS) started a rolling deadline for FHIR compliance in 2020, with fines for institutions that fall behind. As a result, for most U.S.-based healthcare providers, payers, and their technology vendors, the past few years were a headlong race to adopt FHIR. Here are three reasons why FHIR is hugely significant for healthcare technology leaders: It’s a federal mandate from the Centers for Medicare & Medicaid Services. It’s a complex data integration challenge. Legacy systems built before the mid 2010s are not interoperable with the FHIR mandate. FHIR implementation approaches For large organizations with huge data requirements, data architects can experience paralysis from the sheer volume of legacy systems to unwind. These groups have all of their patients’ electronic healthcare record information, payer information and more bound up in legacy systems, none of which is interoperable with FHIR. The second challenge is cloud migration, which can be skirted by organizations using a checkbox compliance approach. In those cases, API layers are used to ingest and serve data to legacy systems, but are not really integrated with the legacy system in real time. The most successful approach to tackling this challenge is not to rewrite, unwind or replace legacy systems completely, but keep them contained. We recommend bringing in an operational data layer that exposes the information in the legacy system and keeps it in sync with the legacy system, but then lands it in an ODL in the FHIR standard. With the FHIR API, patients and providers can interact with data in real time and access records in milliseconds after a diagnosis. Real-time records synced with legacy systems and patients’ private data is protected. Delve into the full conversation below Your browser does not support the audio element. FHIR and the future of healthcare at Humana You don't have to take the rip and replace approach when modernizing your legacy systems with an ODL method. This was a key to successful modernization for Humana, as discussed in the third and final episode in our series. For large enterprises that may have decades’ worth of acquired legacy systems, often pulling similar datasets from disparate databases, the pursuit of modernized interoperability begins to look like an impossible task. Listen to the final episode of our podcast series to here how Humana’s ODL approach met the company’s data velocity requirements, and next steps for personalized healthcare and interoperability at Humana. Listen to the entire conversation below Your browser does not support the audio element. More related FHIR and healthcare resources [ White paper ] Bring the FHIR Inside: Digital Transformation Without the Rip and Replace [ On-demand webinar ] Building FHIR Applications with MongoDB
Introducing Pay as You Go MongoDB Atlas on AWS Marketplace
We’re excited to introduce a new way of paying for MongoDB Atlas . AWS customers can now pay Atlas charges via our new AWS Marketplace listing . Through this listing, individual developers can enjoy a simplified payment experience via their AWS accounts, while enterprises now have another way to procure MongoDB in addition to privately negotiated offers, already supported via AWS Marketplace. Previously, customers who wanted to pay via AWS Marketplace had to commit to a certain level of usage upfront. Pay as you go is available directly in Atlas via credit card, PayPal, and invoice — but not in AWS Marketplace, until today. With this new listing and integration, you can pay via AWS with no upfront commitments . Simply subscribe via AWS Marketplace and start using Atlas. You can get started for free with Atlas’s free-forever tier , then scale as needed. You’ll be charged in AWS only for the resources you use in Atlas, with no payment minimum. Deploy, scale, and tear down resources in Atlas as needed; you’ll pay just for the hours that you’re using them. Atlas comes with a Basic Support Plan via in-app chat. If you want to upgrade to another Atlas support plan , you can do so in Atlas. Usage and support costs will be billed together to your AWS account daily. If you’re connecting Atlas to applications running in AWS, or integrating with other AWS services , you’ll be able to see all your costs in one place in your AWS account. To get started with Atlas via AWS Marketplace, visit our Marketplace listing and subscribe using your account. You’ll then be prompted to either sign in to your existing Atlas account or sign up for a new Atlas account . Try MongoDB Atlas for Free Today!
10 Signs Your Data Architecture is Limiting Your Innovation: Part 2
With the massive amounts of data organizations now ingest, store, and analyze comes a massive responsibility to monitor, manage, and protect it. Unfortunately, many businesses are functioning with little insight into how their data is stored and who is accessing it — and their overly complex data architecture can turn those challenges into Frail security can lead to unnecessary risk, and if you are not in control of your data architecture, the next big compliance offender or data breach victim could be you. These risks — and the time and resources required to address them — make up part of a hidden tax on your innovation. We call it DIRT — the Data & Innovation Recurring Tax . Our experts have identified 10 symptoms that can indicate your business is paying DIRT — read about them all in our white paper 10 Signs Your Data Infrastructure is Holding You Back , and check out Part 1 of this blog series. Here, we highlight two signs of this innovation tax that are all about security. Symptom #3: That last big data breach — or the next one — is on you The more complex your data architecture, the more threat vectors you need to cover and the more complicated and time-consuming it becomes to maintain security. Each data store and application may have its own security framework and requirements — its own access controls, role definition, and login procedures. Each database may in turn be connected with multiple other technologies and vendors, further adding to the time and complexity needed to keep everything secure. That’s a drag on your team: Some 30% of IT managers spend more than 16 hours a month just on patching, and 14% spend more than 48 hours a month. Often, it’s impossible to keep up: 42% of breaches are the result of an attack for which a patch was available but not applied, according to a Ponemon Institute study of IT professionals. On average, 28% of vulnerabilities remain unaddressed. Our Solution: With an application data platform, you have one set of database and data service components that share the same developer experience and the same underlying operational and security characteristics, making it a lot easier to defend. Organizations can use a single overarching security policy and implementation without having to reinvent the wheel every time someone has a new use case for the data. Maintaining audit logs and access is dramatically streamlined. You get both security and speed. Symptom #4: Rampant data duplication makes compliance a nightmare In a modern organization, every part of the business should have access to the data and insights that help optimize performance and meet customer demand. But most data is trapped in silos, each with its own formats, access, and authorization controls. Attempts to address data silo issues often create their own web of separate niche data technologies, each trying to solve the problem. That can create a lot of data duplication — so even your IT leaders may not know who has copies of which data, or even how many copies there may be. That’s obviously a problem for security reasons. It also makes it extremely difficult to comply with regulations such as GDPR and the California Consumer Privacy Act, or to respond effectively to audits. How can you tell your regulators exactly where personally identifying information sits, or where it has been, when you don’t even know how many copies exist? Our Solution: Eliminate silos in the first place by using an application data platform, which addresses many of the use cases that would otherwise spur teams to duplicate data. And, with MongoDB, you can federate queries across multiple sources so you don’t have to move data into different formats. Our next installment will focus on your developers’ time, how it’s spent and the price you pay when they can’t find the time to develop and roll out best-in-class features. For a complete view of DIRT, read our white paper DIRT and the High Cost of Complexity .
MongoDB x Screaming in the Cloud: A Discussion
Held in Las Vegas every winter, AWS re:Invent features booths and exciting new demos from the biggest names in tech; a slate of fun, engaging activities; and inspirational keynotes by thought leaders and pioneers. Along with being one of today’s top tech expos, re:Invent is also the ideal venue for thinkers to meet and exchange ideas. At this year’s conference, Sahir Azam, Chief Product Officer at MongoDB, sat down with Corey Quinn, Chief Cloud Economist at Duckbill Group (and one of the most interesting men in tech), for a deep, wide-ranging conversation. Their chat covers everything from the state of databases today to the true definition of an application data platform. Read on for some highlights and listen to the episode here . How is MongoDB adapting to the cloud? Corey kicks off the talk with a big-picture question: How has MongoDB, a mainstay in the database world, evolved to match the rapidly changing demands of the market? Given the rapid proliferation of databases and related technologies, this question is especially timely. “What do you do these days?” Corey asks. “What is MongoDB in this ecosystem?” “Today, MongoDB has become one of the leading cloud database companies in the world,” Sahir replies. “The majority of [our] business comes from our cloud service. That’s our flagship product.” One database to rule them all? “[That] leads to the obvious question,” Corey continues. “What’s your take on the whole idea of a different database for every problem/customer/employee/API request?” “[Many] customers clearly moved to the cloud because they want to be able to move faster, innovate faster, be more competitive,” Sahir replies. Although it’s impossible for a single database vendor to address every customer need, Sahir also mentions that “cobbling together 15 different databases” forces teams to focus on troubleshooting instead of innovation. Instead, Sahir points out, the ideal database would fit “80% of [an organization’s] use cases, with niche technologies serving as specialized solutions for particular needs.” What is the nature of MongoDB's relationship with AWS? “You mentioned that you are a partner with AWS,” Corey asks. “But how do you address the idea of partnering with a company that also heavily advantages its own first-party services?” Sahir’s reply — that MongoDB has a complex, multifaceted relationship with AWS but not an adversarial one — cites the two companies’ mutual interests and partnerships. “The idea of working with major platform players...being a customer, a partner, and a competitor is something that any organization at our scale and size [has to] navigate,” Sahir explains. “Honestly, there’s a lot more collaboration, both on the engineering side and in the field. We jointly work with customers and get them onto our platform way more often than the world sees.” And much more... Corey and Sahir’s discussion also covers how international customers use MongoDB, how potential users and customers perceive MongoDB, and what’s in store for future MongoDB products. Check out the full podcast!
PeerIslands Cosmos DB Migrator Tool to MongoDB Atlas on Google Cloud
When you’re in the midst of innovating, the last thing you want to worry about is infrastructure. Whether you’re looking to streamline inventory management or reimagine marketing, you need applications that can scale fast and maintain high availability. That’s where MongoDB Atlas on Google Cloud comes in. With MongoDB Atlas’ general-purpose, document-based database, users can free themselves from the hassle of database management, and give back precious time to developers to focus on innovation. Combine these benefits with Google Cloud’s cloud computing power, high availability, and ability to integrate with tools like BigQuery, Dataflow, Dataproc and more, and it’s hard to find a comparable joint solution. In fact, many current Microsoft Azure Cosmos DB users are now considering making the move to MongoDB. Microsoft’s Cosmos DB only supports single partition transactions, has no schema governance and forces developers to work with five different APIs to deliver full application functionality. Conversely, MongoDB Atlas on Google Cloud supports distributed multi-document ACID transactions, includes schema governance, and offers integrated full-text search, auto-archiving, data lakes, and edge-to-cloud data sync. The following blog illustrates how PeerIslands’ Cosmos DB Migrator tool can help users move from Cosmos DB to MongoDB Atlas on Google Cloud. Why PeerIslands PeerIslands is an enterprise-class digital transformation company composed of a team of polyglots who are comfortable across multiple technologies and cloud platforms. As a services firm, PeerIslands is focused on helping customers with both cloud-native development and application transformation. With best-in-the-industry talent, PeerIslands has been working with the MongoDB team to build a suite of solutions around two key objectives: For a customer evaluating MongoDB, how can we rapidly address common questions? Once a customer has chosen MongoDB, how can we reduce time to value by rapidly migrating workloads to MongoDB? With this in mind, PeerIslands developed a suite of tools around schema generation, understanding MongoDB query performance, as well as helping customers understand code changes required for upgrading MongoDB versions. In terms of workload migrations, PeerIslands developed solutions for both homogenous and heterogenous migrations. The company is also contributing to the open source community with a mobile app for enabling MongoDB admins to manage Atlas on the go. PeerIslands' Cosmos DB migration use case The current approach for migrating data from Cosmos DB to MongoDB is to use MongoDB dump and restore. But there are several problems with this approach. It’s fully manual and CLI-based which creates a poor user experience and requires technical resources even for simple migrations. There’s a lack of change capture capability which requires downtime during the duration of migration. For large Cosmos DB migrations, this causes significant issues. The team is also under pressure to deliver the entire migration in a short period of time. Migrations often get delayed as customers have difficulty identifying the right migration window. The Cosmos to MongoDB tool is a “Live Migrate” like tool that helps perform one-time migrations and change data capture from Cosmos DB (MongoDB model) to MongoDB Atlas and minimizes downtime requirements associated with migrations. The tool is fully GUI-based and nearly everything is automated. All the tasks for infrastructure provisioning, dump & restore, change stream listeners and processors have all been automated with a graphical user interface (GUI). The Cosmos to Mongo migration tool uses native MongoDB tools and the performance is similar to native tools. For change capture, we leverage the native MongoDB change stream APIs. A high level view of the solution is provided in figure 1 below: Figure 1: Solution Map Migration steps: Migration configuration: Provide the name of the migration task, source Cosmos DB details, and target MongoDB details. The tool supports key vault integration as well. Migration infrastructure provisioning: Provide migration infrastructure details required for creating the VM (Virtual Machine) including location, type of VM instance, etc. Migration execution: Allow for automation of the migration once the configuration is complete. The migration is executed in 3 steps: backup, restore and change event processing. As a user, you can initiate the backup process. The change event listener is started in parallel with the backup process and captures all the changes. Once the backup is complete, the user can restore the initial data and then perform change event processing to apply all the changes to MongoDB. Migration validation: The tool also provides facilities for validating the migration. Users can view the total number of documents on both source Cosmos DB collection and target MongoDB collection. They can also compare random documents picked up from Cosmos DB and MongoDB side by side and validate whether the data elements have been loaded correctly. For a more detailed demo and description of events, watch the following video: Migrating to a new database can feel daunting at first, but PeerIslands Cosmos DB migrator makes it easy. Major concerns like delays and downtime are eliminated from the process, helping you run your business smoothly and reap the benefits of MongoDB more quickly. And with PeerIslands suite of tools, you can rapidly address MongoDB-specific questions and accelerate time to value. Reach out today to get started
Uncover hidden truths with Interactive Filtering
Atlas Charts has been updated with greater dashboard interactivity! With Interactive Filtering, users can click on charts to highlight and filter their dashboard data to their heart’s content. Interactive dashboards support better decision making Data visualisations are meant to be more than just a pretty collection of charts. They are a valuable decision making tool. At MongoDB, we believe that users should not only be able to manipulate their data but explore the significance of data changes in their dashboards to aid in making faster decisions. Last year, we added the ability to interact with charts and dashboards using Dashboard Filtering . This year, we are taking it to the next level. We are very excited to release ‘Interactive Filtering’, allowing users to interact with a particular chart to filter data in other charts. How does Interactive Filtering work? So let’s use an example to understand how this feature works: We have built a dashboard with multiple charts using the ‘Movies’ sample data source and added dashboard filters for the ‘languages’ and ‘rated’ fields. Previously, you could only interact with these filters by checking or unchecking options in the filter pane. While that option remains available, Interactive Filtering allows you to explore your data directly from the charts. So when you click on chart elements, such as bars, category labels or legend entries, it is a faster way to filter your dashboard. Chart elements are clickable if they are built with fields that are linked to dashboard filters. When filter fields are used on a chart’s category channels, we now highlight the requested data point but continue to show the other chart data, making it easy to see how the values compare. Looking at the dashboard below, if you want to know how many Spanish movies are released or the average metacritic score of Spanish movies, then you can highlight ‘Spanish’ by clicking the ‘Spanish’ bar or the ‘Spanish’ label in the ‘Movies by Language’ chart. The other charts in the dashboard get filtered to show only data based on the Spanish language. You can also use the ‘languages’ dashboard filter to achieve the same result. Interact by clicking directly in your dashboard Interact by modifying your dashboard filters How do you know what Charts are highlighted vs filtered? We now show a handy filter and highlight icon to indicate whether a chart is being filtered, highlighted, or both. What if you want to go back to the previous state before you started clicking? Simple, just click anywhere on the empty area of a chart and the changes are rolled back. If you were to use the dashboard filter pane to highlight or filter the dashboards, we save the state. Think of the dashboard filter pane changes as a persistent state and click changes on a chart as a temporary state. Depending on the dashboard filters selected, you also have the option to make a chart in the dashboard filter or highlight, filter only or ignore the dashboard filters completely. Try Interactive Filtering today With ‘Interactive Filtering’ the possibilities to manipulate and explore your data are limitless. Try the new highlighting functionality today by creating dashboard filters for your key fields and uncover the hidden answers in your data. New to Charts? You can start now for free by signing up for MongoDB Atlas , deploying a free tier cluster and activating Charts. Have an idea on how we can make MongoDB Charts better? Feel free to leave an idea at the MongoDB Feedback Engine .
Export Queries to Your Favorite Programming Languages with MongoDB for VS Code
In MongoDB for VS Code 0.7.0 we introduced the ability to export queries and aggregations that you’ve drafted in a playground to your favorite programming language. Going from prototype to code is now easier than ever. At MongoDB, we spend a lot of time thinking about how we can make developer workflows as easy as possible. This is why last year we released MongoDB for VS Code , an extension that lets you easily work with MongoDB from your VS Code environment. Since that first release, we have been looking at how developers use the extension and we have been talking to a lot of our users to understand what else we could add to make them even more productive. Everybody we talked to, really enjoyed the ability to navigate and explore their data and keep an eye on their MongoDB collections right next to their application code. We also learned that a lot of our users take advantage of Playgrounds to prototype their queries and make sure they return the results they expect with the performance they expect. Other developers go even further. They use Playgrounds as a way to keep track of all the queries and aggregations their application code runs in one central place and they store playground files in git repositories together with their code. This is a great way to document what the code is supposed to do when it talks to MongoDB so that other engineers on the team can always have an up-to-date birds-eye view of how the application interacts with the database. With this use case in mind, we found an opportunity to streamline the workflow even further. We asked ourselves: what if, Playground files could be used as the source of truth for all queries and aggregations and the actual application code could just be automatically generated from there? With the most recent release of MongoDB for VS Code, we took the first step in this direction. You can now select queries and aggregations within your Playground files and translate them into your favorite programming language. We currently have support for Node.js, Python, Java, and C#. With this new addition to our VS Code extension, we are hoping to make the process of designing and testing your queries and getting them into your application easier and more effective. If you try out this new functionality and you want to share your feedback, feel free to submit ideas or feature requests on our feedback portal or to reach out directly to me on Twitter .