Whether it is enterprise, government, or science and research, big data and analytics projects require processing large volumes of data. Data ingestion, movement, and transformation operations consume significant time and resources. This is exacerbated by the fact that in today’s rapidly shifting ecosystem, data comes from a wide variety of sources: structured and unstructured, logs, streaming data, and more. Azure Data Factory (ADF) is an Azure service that allows you to create pipelines for automatically ingesting, moving, and processing large volumes of data. In this session, you will learn the basics of ADF, how to create and schedule pipelines, process data from various sources, and how to create custom activities with C#.
Joel Cochran is currently serving as a Technical Evangelist on the Microsoft Civic Technology Services team. Joel was born in Virginia, grew up in Iowa, and graduated from James Madison University in Harrisonburg, VA. After serving in the United States Army, he moved to Florida where he fell backwards into a job at a software company. Before long he was a software developer on the IBM AS/400 platform. After moving back to Virginia a few years later, he worked for a micro-ISV writing software for local governments. This period led him through a wide range of technologies such as RPG, DB2, Java, Perl, PHP, MySQL, Linux, and more. In 2003, Joel discovered C#, .NET, and SQL Server, and has been dedicated to the Microsoft stack ever since. Over the years he has ranged through a variety of web, desktop, and database development tools and environments. An accomplished writer and speaker, Joel loves to teach and was a 6-time Microsoft MVP prior to joining the company. He lives in Harrisonburg, VA, with his family, and is an avid pool player.