Accelerating Data Engineering Pipelines
3
Lessons
8h
Duration
English
Language
Prerequisites:
- Intermediate knowledge of Python (list comprehension, objects)
- Familiarity with pandas a plus
- Introductory statistics (mean, median, mode)
Technologies: pandas, cuDF, Dask, NVTabular, Plotly
Assessment Type: Skills-based coding assessments evaluate your ability to efficiently filter through millions of data points in the context of an interactive dashboard.
Certificate: Upon successful completion of the assessment, you’ll receive an NVIDIA DLI certificate to recognize your subject matter competency and support your professional career growth.
Hardware Requirements: You’ll need a desktop or laptop computer capable of running the latest version of Chrome or Firefox. You’ll be provided with dedicated access to a fully configured, GPU-accelerated workstation in the cloud
Data engineering is the foundation of data science and lays the groundwork for analysis and modeling. In order for organizations to extract knowledge and insights from structured and unstructured data, fast access to accurate and complete datasets is critical. Working with massive amounts of data from disparate sources requires complex infrastructure and expertise. Minor inefficiencies can result in major costs, both in terms of time and money, when scaled across millions to trillions of data points.
In this workshop, we’ll explore how GPUs can improve data pipelines and how using advanced data engineering tools and techniques can result in significant performance acceleration. Faster pipelines produce fresher dashboards and machine learning (ML) models, so users can have the most current information at their fingertips.
Learning Objectives
- How data moves within a computer. How to build the right balance between CPU, DRAM, Disk Memory, and GPUs.
- How different file formats can be read and manipulated by hardware.
- How to scale an ETL pipeline with multiple GPUs using NVTabular.
- How to build an interactive Plotly dashboard where users can filter on millions of data points in less than a second.
Learning Path
- Meet the instructor.
- Create an account at courses.nvidia.com/join
15 mins
- Pandas
- CuDF
- Dask
15 mins
- Transform raw json into analysis-ready parquet files
- Learn how to quickly add features to a dataset, such as Categorify and Lambda operators
60 mins
- Learn how to use descriptive statistics and plots like histograms in order to assess data quality
- Learn effective memory usage, so users can quickly filter data through a graphical interface
Users are complaining that the dashboard is too slow. Apply the techniques learned in class to find and eliminate efficiencies in the backend code
60 mins
- Review key learnings and answer questions.
- Complete the assessment and earn your certificate.
- Complete the workshop survey.
- Learn how to set up your own AI application development environment.
