Palantir has often been described as a secretive company. There is some truth to this. For many years, we primarily serv

At Airbnb, we’ve always had a data-driven culture. We’ve assembled top-notch data science and engineering teams, built industry-leading data infrastructure, and launched numerous successful open source projects, including Apache Airflow and Apache Superset. Meanwhile, Airbnb has transitioned from a startup moving at light speed to a mature organization with thousands of employees. During this transformation, Airbnb experienced the typical growth challenges that most companies do, including those that affect the data warehouse. This post explores the data challenges Airbnb faced during hyper growth and the steps we took to overcome these challenges.,Technology companies should be scrutinised, especially when their technology is used by governments. To that end, we hope this series of blog posts will provide readers with more insight into our business and a deeper understanding of some of the key features of our technology. In our next post, we will discuss Purpose-Based Access Controls. Please feel free to reach out at [email protected].,Prior to the Data Quality Initiative described in this post, data asset ownership was distributed mostly among product teams, where software engineers or data scientists were the primary owners of pipelines and datasets. However, data ownership responsibilities were not clearly defined — this was a bottleneck when issues arose.,We build digital infrastructure for data-driven operations and decision making. Our products serve as the connective tissue between an organisation’s data, its analytics capabilities, and operational execution. Palantir’s platforms tie these together by bringing the right data to the people who need it, allowing them to take data-driven decisions, conduct sophisticated analytics, and refine operations through feedback. We license this software to organisations, who receive secure and unique instances of our platforms in which to conduct their own work on their own data.,If you’re lucky enough to have $1,000 left over at the end of every month, consider it a sacred responsibility to use that money wisely. You can still have a great life in the process. Getting to a million bucks isn’t going to, in and of itself, give you a great life. But it can certainly open up a lot of doors that not being wealthy leaves closed.,This blog post will be the first in a series, Palantir Explained, that will explore a range of topics, including our approach to privacy, security, AI/ML safety and more. We hope you find these blogs useful and informative; and we welcome feedback. If you find these blog posts inspiring and would like to join us in solving the challenges described, have a look at the opportunities we have available — we’d love to hear from you.,Unlike many tech companies, our business model is not based on the monetisation of personal data. We do not collect, store, or sell personal data. We don’t use personal data to train proprietary AI or machine learning models to share or resell to other customers. We never facilitate the movement of data between clients, except where those specific clients have entered into an agreement with each other.,As Airbnb grew from a small start-up to the company it is today, many things have changed. The company has ventured into new business areas, acquired numerous companies, and significantly evolved product strategy. Meanwhile, the requirements on our data have also changed. For instance, leadership has set high expectations for data timeliness and quality, and increased focus on cost and compliance. To ensure that we continue to meet these expectations, it was apparent that we needed to make sizable investments in our data. These investments centered around addressing areas related to ownership, data architecture, and governance.,Technology companies should be scrutinised, especially when their technology is used by governments. To that end, we hope this series of blog posts will provide readers with more insight into our business and a deeper understanding of some of the key features of our technology. In our next post, we will discuss Purpose-Based Access Controls. Please feel free to reach out at [email protected].,With regards to customer data, we act as a data processor, not a data controller. Our software and services are used under direction from the organisations that license our products: these organisations define what can and cannot be done with their data; they control the Palantir accounts in which analysis is conducted; and any Palantir engineers that assist them in their work follow these directions. We do not and cannot reuse or transfer our clients’ data for our own purposes. Attempting to profit from customer data in this way would be illegal and would undermine the trust that is necessary to work in the sensitive environments in which we have built our business.,Now that we serve clients in a wider range of sectors, we have an opportunity to be more open. This is particularly true for sectors like healthcare, where Palantir’s software is used to process personal data. People have a right to understand how Palantir technology works, and how our customers use it.,This infrastructure helps organisations bring the right data together at the right time to answer complex questions and make intelligent decisions. This is particularly valuable when existing systems are fragmented, and essential information is held in silos that can’t communicate with each other. Healthcare organisations, for instance, have used our software to tackle challenges like efficiently allocating PPE supplies when thousands of hospitals across the country have radically different and constantly changing levels of supply and demand for each item of PPE.,Most of the pipelines that were constructed during the company’s early days were built organically without well-defined quality standards and an overarching strategy for data architecture. This led to bloated data models and placed an outsized operational burden on a small group of engineers.,These platforms are used in some of the most sensitive and secure information environments in the world, necessitating world-class data protection and governance. As such, we have an exceptionally strong information security record.,Common misconceptions recur, particularly around the assumption that Palantir can use or transfer client data for its own purposes, or can join data from different clients together to sell on. This is not how we operate and never has been. In this blog post, we will explain more about the nature of Palantir’s business.