Senior Software Engineer (Hybrid/ In-office) - Military Veterans
at Hearst
As an organization, Black Book is open, transparent, and very customer focused. Leaders in the organization consistently showcase the following key attributes and look to build teams that embrace these qualities every day:
- Shift Left Mindset
- Constant desire for self-improvement and growth
- Embracing change and ambiguity
- Strong sense of ownership, focusing on accountability
- Oriented around the internal and external customer experience
- Foster and celebrate an inclusive environment
Black Book: Black Book provides vehicle information and valuations to dealerships, financial institutions, fleet owners and other companies in the automotive industry. We develop APIs and web and mobile applications that provide insights on a variety of motor vehicles of various markets.
Web Crawler Team: We are seeking a highly skilled software engineer to join Black Book's Web Crawler team. The main responsibility of the team is to build, enhance, support, and innovate on a key automotive retail listings web crawler project. The ideal candidate will have extensive experience with Scrapy and Python for web scraping, and strong knowledge of PostgreSQL and SQLAlchemy for database management. Proficiency in browser automation using Playwright and Selenium is crucial, alongside expertise in Redis for data storage, and ElasticMQ for local message queuing and AWS SQS in the cloud. Familiarity with Kubernetes and Docker Desktop for deployment, and competency in using Poetry for dependency management, are also required. This is an exciting opportunity for someone who thrives in a fast-paced environment and is enthusiastic about technological innovation and problem-solving.
You
You are comfortable working with developers and members outside your core team to achieve success. You enjoy taking opportunities to fix problems, mentor your peers, and like to step outside your comfort zone to develop your skillset.
Responsibilities:
- Develop and maintain secure, efficient, and scalable web scraping solutions using modern technologies.
- Analyze, design, and implement strategies for robust data extraction and processing systems.
- Ensure compliance with web scraping ethics, privacy, and legal standards.
- Troubleshoot, debug, and resolve complex issues in web crawling processes effectively.
- Optimize and refactor crawling solutions for enhanced performance and scalability.
- Monitor, maintain, and support current web crawling systems to ensure reliability and efficiency.
- Collaborate with cross-functional teams to integrate data solutions with stakeholder needs.
- Engage in rigorous peer review and code review sessions to ensure high code quality standards.
- Develop and execute comprehensive automated testing frameworks for web scraping solutions.
- Stay updated on the latest trends and advancements in web scraping and data extraction technologies.
- Proactively address and resolve challenges with innovative and agile solutions.
- Demonstrate ownership and accountability for project milestones and deliverables.
- Provide clear documentation and training for team members on web crawling solutions and best practices.
- Bachelor's degree in computer science or equivalent work experience in software development.
- 5+ years of experience in developing web-based solutions, proficient in Python and modern technologies.
- Demonstrable expertise in web scraping frameworks, such as Scrapy, and browser automation tools like Selenium and Playwright.
- Solid understanding and practical experience with PostgreSQL for relational database management.
- Skilled in deploying and managing applications in Docker and Kubernetes environments.
- Proficiency in utilizing cloud-based services, specifically AWS SQS, with exposure to ElasticMQ for local development.
- Experience in implementing and maintaining Continuous Integration/Continuous Deployment pipelines.
- Seasoned in both unit and integration testing methodologies to ensure code quality and reliability.
- At least 2 years of working in agile environments, promoting collaboration and continuous improvement.
- Experience with version control systems, particularly Git, to manage and track code changes.
- Excellent communication skills - the ability to think creatively and adapt the message to the audience
- Strong problem-solving skills, proactive, adaptable, with the ability to take initiative and drive projects forward.
- Experience working with Angular, React, Next.js, and NestJS.
Lawrenceville, GA
Hearst is one of the nation’s largest global, diversified information, services and media companies. Hearst has been innovating for more than a century, leading with purpose, integrity and a culture of care, with a mission to inform audiences and improve lives. The company’s diverse portfolio includes global financial services leader Fitch Group; Hearst Health, a group of medical information and services businesses; Hearst Transportation, which includes CAMP Systems International, a major provider of software-as-a-service solutions for managing maintenance of jets and helicopters; ownership in cable television networks such as A&E, HISTORY, Lifetime and ESPN; 35 television stations; 24 daily and 52 weekly newspapers; digital services businesses; and more than 200 magazines around the world. Hearst is always moving forward, investing in healthcare solutions to improve patient outcomes and technology that curbs emissions; providing vital analysis, data and software to the global financial services industry; delivering important service and investigative journalism; and inspiring audiences with sports and entertainment programming. With a commitment to maintaining the highest quality in its products and services, Hearst is dedicated to serving the communities it operates in, both civically and philanthropically.