Meanwhile, the company built Minerva, a widely-adopted platform that catalogs metrics and dimensions and computes joins

The company’s initial analytics foundation, “core_data”, was a star schema data model optimized for ease-of-use. It was built and owned by a central team, and incorporated numerous sources — often across different subject areas. This model worked extremely well in 2014; however, it became more and more difficult to manage as the company grew. Based on this learning, it was clear that our future data model should be designed thoughtfully and avoid the pitfalls of centralized ownership.

We also committed to a decentralized organizational structure composed of data engineering pods reporting into product teams (as opposed to a single centralized Data Eng org). This model ensures data engineers are aligned with the needs of consumers and the direction of product, while ensuring a critical mass of engineers (3 or more). Team size is important for providing mentorship/leadership opportunities, managing data operations, and smoothing over staffing gaps.

To complement the distributed pods of data engineers, we founded a central data engineering team that develops data engineering standards, tooling, and best practices. The team also manages global datasets that don’t align well with any of the product teams.

Data Architect Working Group — Composed of senior data engineers from across the company. Responsible for making major architectural decisions, and conducting reviews for Midas certification (see below).

Data Engineering Leadership Group — Composed of data engineering managers and our most senior Individual Contributors. Responsible for organizational and hiring decisions.

Once momentum on the Data Quality initiative reached a critical point, leadership realigned the company’s limited data engineering resources to kickstart the project. This was sufficient to unblock progress on Airbnb’s most critical data; however, it became obvious that we needed to unite and substantially grow the data engineering community at Airbnb. Below are changes we made to facilitate progress.

The next step was to align on a common set of architecture principles and best practices to guide our work. We provided comprehensive guidelines for data modeling, operations, and technical standards for pipeline implementation, which are discussed below.

For several years, Airbnb did not have an official Data Engineer role. Most data engineering work was done by data scientists and software engineers who were recruited under a variety of different monikers. This misalignment made hiring for data engineering skill sets very challenging, and created some confusion with respect to career progression. To resolve these issues, we reintroduced the role “Data Engineer” as a specialization within the ranks of the Engineering organization. The new role requires Data Engineers to be strong across several domains, including data modeling, pipeline development, and software engineering.

http://live247.gruposio.es/udo/v-ideos-brest-v-nice-v-fr-fr-1qpj-14.php

http://vert.actiup.com/vyw/v-ideos-brest-v-nice-v-fr-fr-1ysj-27.php

http://vert.actiup.com/vyw/v-ideos-brest-v-nice-v-fr-fr-1ofp-23.php

http://news24.acaps.cat/gms/v-ideos-metz-v-bordeaux-v-fr-fr-1lhh-21.php

http://news24.acaps.cat/gms/video-metz-v-bordeaux-v-fr-fr-1lpf-6.php

http://vert.actiup.com/vyw/videos-brest-v-nice-v-fr-fr-1opr-8.php

https://assifonte.org/media/sja/Video-nantes-v-rennes-v-fr-fr-1hwg-14.php

http://news24.acaps.cat/gms/video-AEL-Limassol-AEK-Larnaca-v-en-gb-1yem30122020-.php

http://news24.acaps.cat/gms/videos-AEL-Limassol-AEK-Larnaca-v-en-gb-1bog30122020-22.php

http://vert.actiup.com/vyw/Video-brest-v-nice-v-fr-fr-1fkr-11.php

https://assifonte.org/media/sja/Video-nantes-v-rennes-v-fr-fr-1gdg-15.php

http://vert.actiup.com/vyw/videos-Brest-OGC-Nice-v-en-gb-1aux30122020-.php

http://vert.actiup.com/vyw/v-ideos-Brest-OGC-Nice-v-en-gb-1joc-11.php

http://news24.acaps.cat/gms/Video-AEL-Limassol-AEK-Larnaca-v-en-gb-1xyi-27.php

http://stream88.colomboserboli.com/urf/videos-Asteras-Tripolis-Olympiacos-v-en-gb-1kal-15.php

https://assifonte.org/media/sja/videos-nantes-v-rennes-v-fr-fr-1pyz-1.php

http://wap.dentisalut.com/ltc/video-Metz-Bordeaux-v-en-gb-1ovr-8.php

http://stream88.colomboserboli.com/urf/videos-Metz-Bordeaux-v-en-gb-1bcd30122020-.php

https://assifonte.org/media/sja/Video-nantes-v-rennes-v-fr-fr-1xfi-13.php

http://wap.dentisalut.com/ltc/videos-Metz-Bordeaux-v-en-gb-1voi-8.php

thing wrong with creating a vlog channel that documents your life or creating random videos for the sake of being creative, focusing on a specific niche that commands a higher price per click from advertisers will make it easier to earn more from fewer views and a smaller subscriber count.

In early 2019, the company made an unprecedented commitment to data quality and formed a comprehensive plan to address the organizational and technical challenges we were facing around data. Airbnb leadership signed off on the Data Quality initiative — a project of massive scale to rebuild the data warehouse from the ground up using new processes and technology.

Most of the pipelines that were constructed during the company’s early days were built organically without well-defined quality standards and an overarching strategy for data architecture. This led to bloated data models and placed an outsized operational burden on a small group of engineers.

The following sections detail the specific approach that was taken to move this effort forward, with specific focus on our data engineering organization, architecture and best practices, and the processes we use to govern our data warehouse.

We revamped our hiring process for data engineers, and allocated aggressive headcount towards growing our data engineering practice. We paid particular attention to bringing in senior leaders to provide direction as we make decisions that will affect the organization in the years to come. This is an ongoing effort.

We created new communication channels to better connect the data engineering community, and established a framework for making decisions across the organization. We created the following groups to address these gaps:

In addition to needing to lay out an overarching strategy for data architecture, Airbnb also needed a centralized governance process to enable teams to adhere to the strategy and standards.