Data Solutions Engineer/Manager

About the Company

• YipitData is the leading market research firm for the disruptive economy and recently raised $475M from The Carlyle Group at a valuation of over $1B.
• We analyze billions of data points every day to provide accurate, detailed insights on ridesharing, e-commerce marketplaces, payments, and more. Our data team uses proprietary technology to identify, license, clean, and analyze the data many of the world’s largest investment funds and corporations depend on.
• For three years, we have been recognized as one of Inc’s Best Workplaces. We are a fast-growing technology company backed by Norwest Venture Partners and The Carlyle Group. We cultivate a strong people-centric culture focused on mastery, ownership, and transparency.

We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, marital status, disability, gender, gender identity or expression, or veteran status. We are proud to be an equal-opportunity employer.

About this role

We are actively seeking a Data Solutions Engineer to join our dynamic ETL Engineering team. At the core of ETL Engineering’s mission is the creation of best-in-class tooling to build highly performant, reliable, and cost-effective data pipelines. This team plays a pivotal role in maintaining and optimizing critical data pipelines, including high-volume 1st party and 3rd party data feeds that power all of our products, as well as managing our internal web-scraper ingestion applications.

Additionally, ETL Engineering serves as the gold standard for all other YipitData analyst teams, offering support through training, refactoring, and enhancement for their web scrapers and data pipelines. This high-impact, high-visibility team is instrumental to the success of our rapidly growing business. This is a unique opportunity to be the first hire in this team, with the potential to build and lead the team as their responsibilities expand.

• Candidates should be available and willing to work during East Coast business hours. (till 5 pm EST)

As Our Data Solutions Engineer You Will

• Report directly to the Director of ETL Engineering, who will provide significant, hands-on training on cutting-edge data tools and techniques.
• Become an expert at solving complex data pipelines issues using PySpark and SQL.
• Become an expert in understanding web API requests.
• Deeply learn Databricks and Spark so you can analyze inefficient and failing data pipelines.
• Provide active data pipeline, data access, and other technical support for our analysts.
• Collaborate with analyst stakeholders to help refactor their pipelines for reliability, and efficiency, and to enforce best practices.
• Create documentation, architecture diagrams, and other training materials for analyst stakeholders.
• After ~6 months of learning the job, you will work with the Director of ETL Engineering to build out a Data Solutions Engineering team and hire 2-3 other team members.
• You will manage this team and its individuals as a “player-coach,” while also continuing to develop your technical skills and knowledge.

You Are Likely to Succeed If

• You hold a Bachelor’s or Master’s degree in Computer Science, STEM, or a related technical discipline.

• You have 5+ years of experience as a Data Engineer, Data Analyst, or other technical function, as well as 1-2+ years of management experience, plus experience building out and training a team (e.g., you have managed a team of at least 3 people).

• You are excited about solving data challenges and learning new skills.

• You have some understanding of working with data or building data pipelines.

• You are comfortable working with large-scale datasets

• You are eager to constantly learn new technologies.

• You understand web requests, RESFTUL APIs, and the Dom.

• You are a self-starter who enjoys working collaboratively with stakeholders.

• You have exceptional verbal and written communication skills.

• Nice to have: Experience with Databricks, Snowflake, PySpark or Pandas. Playwright/ Selenium, BI Tools.

For You

• You’ll work with New York based team
• You will have the opportunity to acquire new skills through an internal training program
• Flex PTO for any reason, including sick days (no specified limits), flexible work schedule
• Personal laptop
• Health and wellness package
• Remote work