MDS Newsletter #99

Ayushi Rathod -

Want to know what it's like to be just an inch closer to a century? Ask us! We're publishing our 99th edition of the MDS newsletter, and as we do, we're looking back at the incredible journey we've had. We're excited to celebrate how far we've come, and grateful you've been with us on this journey! 💜 Now let's dive into this edition 👇🏻

  • Hopsworks: Hopsworks is a cloud-based big data platform that uses artificial intelligence and machine learning technologies. It offers a subscription-based model with an Enterprise Feature Store that provides seamless collaboration and real-time performance. Users can easily build and manage pipelines for batch and real-time data, and transition to independent feature, training, and inference pipelines for better collaboration and operational efficiency.
    Hopsworks has raised a total of $13.8M in funding over 3 rounds. Their latest funding was raised on June 29, 2023, from a Series B round.
  • Okera: Okera is an AI-driven data governance platform designed for enterprise-scale data access management, enabling users to engage with company data based on their roles, using preferred tools while following regulatory access limitations. The platform provides a common retrieval platform that includes cataloging services and allows users to access data through self-service with consistent schemas and familiar interfaces, without compromising data security or privacy.
  • Lacework: Lacework is a cloud-based platform that provides complete visibility across a cloud environment and detects any threats, vulnerabilities, misconfigurations, or unusual activities. It enables users to innovate rapidly with security by identifying potential risks and threats to their cloud environment. The Polygraph Data Platform automatically learns and understands how the environment should operate and immediately alerts users if there is any deviation. By utilizing Lacework, businesses can stay one step ahead and ensure their cloud environment is secure

    Here are the data tools of Lacework:
\

Good reads and resources

  • Slowly Changing Dimensions with Dynamic Tables: Learn how to implement Slowly Changing Dimensions type 2 effectively using Dynamic Tables in Snowflake data warehouse by Sasha Lionheart. SCD2 represents changes in dimension data over time, while dynamic tables offer a practical solution to manage dimension data versions, capture history, and trace lineage. Sasha demonstrates how to create and populate a dynamic table, using a merge statement and various DDL commands. Moreover, she covers different scenarios for managing SCD2 changes, such as inserting new dimension rows, updating current rows, and handling data deletes. Finally, she highlights the importance of selecting the right strategy to manage SCD2 tables based on use case requirements and the available resources.
  • Automating DBT + Airflow: What is the benefit of automating the process of running dbt models using Airflow and how can it be achieved step-by-step? This article by Alex Driedger addresses the benefits of automating dbt using Airflow and provides a step-by-step approach to implement it. Alex explains that dbt is an excellent option for data modeling and transformation but requires manual effort. Airflow, an open-source platform for scheduling and authoring data pipelines can automate the process of running dbt models. He offers code snippets that guide readers through the automation of dbt using Airflow, resulting in significant benefits and time savings for data teams.

Upcoming data events, summits and webinars

  • MDS Fest is a community-led celebration of ideas and perspectives on the modern data stack. The event will take place from August 21-25, 2023 and will be held virtually, and globally streamed online. The main aim of this event is to provide a platform for data community members to share their knowledge and insights with each other. The talks and presentations at the event will cover various topics related to modern data stacks. The MDS fest'23 hopes to inspire and bring the data community closer together. Detailed schedule of the event can be found on the website here.
  • Possible Singapore's three-day event, Fuel the Future, on 21-23 August 2023 at Shangri-La Singapore, explores new horizons of innovation with generative AI and data harmony. Attendees will discover trusted AI's capability to enhance their operations, generate customer interest, and add significant value to their business. The event comes with hands-on training and interactive breakout sessions with the expertise of analytics, data, and cloud practitioners. Register today to discover new business possibilities. Website link here.

MDS Jobs

  • Sigma is hiring Senior Analytics Engineer
    Location: USA
    Stack: Fivetran, Snowflake, dbt
    Apply here
  • DataDog is hiring Senior Data Analyst, Product
    Location: New York, USA
    Stack: dbt, Snowflake, Airbyte, Metabase
    Apply here
  • Qogita is hiring BI Analyst
    Location: Netherlands, UK
    Stack: Looker, Fivetran, Snowflake, dbt
    Apply here

Just for fun 😀

Are you always hungry for more information and updates about the ever-evolving world of data?

Well, you're in luck! By following us on LinkedIn and Twitter, you'll gain access to all the latest and greatest data content!

But wait, there's more! We want to hear from you - rate us here and let us know how we're doing.

Love it | It's great |  Good | Okay-ish | Meh

We welcome any suggestions, articles you would like us to showcase, or data engineering job listings that you may have. Don't hesitate to get in touch with us and we would be delighted to incorporate your input into our next edition.

About Moderndatastack.xyz‌‌ - We're building a platform to bring together people in the data community to learn everything about building and operating a Modern Data Stack. It's pretty cool - do check it out :)