Sr Engineer - Observability

Job Id: R0000376986

The pay range is $95,000.00 - $171,000.00

Pay is based on several factors which vary based on position. These include labor markets and in some instances may include education, work experience and certifications. In addition to your pay, Target cares about and invests in you as a team member, so that you can take care of yourself and your family. Target offers eligible team members and their dependents comprehensive health benefits and programs, which may include medical, vision, dental, life insurance and more, to help you and your family take care of your whole selves. Other benefits for eligible team members include 401(k), employee discount, short term disability, long term disability, paid sick leave, paid national holidays, and paid vacation. Find competitive benefits from financial and education to well-being and beyond at https://corporate.target.com/careers/benefits.

About us:

Working at Target means helping all families discover the joy of everyday life. We bring that vision to life through our values and culture. Learn more about Target here.

Team Overview:

Join the Observability Team at Target, where you'll be part of a dynamic and courageous group responsible for the heartbeat of our infrastructure. Our platform supports over 11,000 unique applications, with over 3,000 users logging in daily. We're operating at an immense scale:

• 431 billion metrics processed daily
• 93 billion logs ingested and analyzed each day
• Over 120 million queries run daily
• Managing and storing an astounding 500 TB of data per day

We're not just users but active contributors to the open-source community, heavily involved with projects like OpenTelemetry, GoAlert, Grafana, and ClickHouse. Our team specializes in telemetry instrumentation, Telemetry processing, and storage, leveraging cutting-edge technologies in Observability.

As a Sr. Engineer in the Observability team, you will play a pivotal part in building a new Alert Notification capability that will serve as a critical component for the entire company. This system will deliver over 300,000 alerts configured across 50,000 dashboards to various contact points like Slack, email, and more, managing alerts similarly to Prometheus Alert Manager. The ideal candidate is proficient in Golang, API development, and Kubernetes, with a strong understanding of caching techniques and distributed systems. You should be a self-starter with an extreme ownership attitude, ready to collaborate in a large, distributed team environment that values inclusivity, courage, and curiosity.

As a Sr. Engineer at Target, you serve as a specialist in the engineering team that supports the product. You help develop and gain insight in the application architecture. You can distill an abstract architecture into concrete design and influence the implementation. You show expertise in applying the appropriate software engineering patterns to build robust and scalable systems. You are an expert in programming and apply your skills in developing the product. You have the skills to design and implement the architecture on your own, but choose to influence your fellow engineers by proposing software designs, providing feedback on software designs and/or implementation. You show good problem solving skills and can help the team in triaging operational issues. You leverage your expertise in eliminating repeat occurrences

Job Responsibilities:

- Develop Backend Services: Design, implement, deploy and maintain backend services in Golang for Alert notification infrastructure.

- API Development: Create secure APIs and manage them to facilitate seamless integration with various systems and services.

- Kubernetes Proficiency: Deploy, scale, and manage applications using Kubernetes to ensure high availability and optimal performance.

- Building robust distributed systems: Design highly available Alert notification infrastructure that can scale horizontally and provide deterministic behavior.

- Build Caching systems: Implement effective caching techniques to optimize system performance, scalability, and reliability.

- Collaboration: Work closely with cross-functional teams, embracing a one-team culture to ensure alignment and integration across platforms.

- Inclusivity and Teamwork: Foster an inclusive environment where all team members feel valued and empowered to contribute.

- Curiosity and Innovation: Approach challenges with curiosity, seeking innovative solutions and being open to learning new technologies.

- Courage and Ownership: Demonstrate courage in tackling complex problems and take full responsibility for your work, from inception to deployment and maintenance.

What We Offer:

- Impactful Work: Contribute to projects that process billions of metrics and logs daily, directly influencing the stability of thousands of applications.

- Innovative Environment: Be part of a team that's at the cutting edge of technology, leveraging AI and contributing to open-source communities.

- Professional Growth: Opportunities for continuous learning, development, and career advancement.

- Inclusive Culture: Work in a supportive environment that values diversity, inclusivity, and a one-team spirit.

- Collaborative Team: Join a group that encourages curiosity and courage, fostering innovation and shared success.

Core responsibilities of this job are described within this job description. Job duties may change at any time due to business needs.

About you:
• 4 year degree or equivalent experience
• 5+ years of software development experience

• Tech skills:

• Proficiency in Golang is preferred. Other compiled languages like Java, C++, C, or any C family language are acceptable.
• Strong experience with API development and integration.
• Solid understanding of Kubernetes and containerization technologies.
• Familiarity with alert notification systems and tools like Prometheus Alert Manager as an example.
• Knowledge of caching techniques, distributed systems and performance optimization.
• Experience with open-source technologies such as OpenTelemetry, Kafka, GoAlert, Grafana, InfluxDB, Elastic and ClickHouse.

• Soft skills:

• Excellent problem-solving abilities and attention to detail.
• Strong communication skills, both written and verbal.
• Ability to thrive in a large team environment.
• Inclusivity: Commitment to fostering an inclusive and collaborative team culture.
• Curiosity: to learn and explore new technologies and methodologies.
• Courage: Willingness to take on challenges and advocate for innovative solutions.
• Self-Starter: Proactive approach to identifying and solving problems.
• Extreme Ownership: Demonstrated responsibility and accountability for tasks and projects.

This position will operate as a Hybrid/Flex for Your Day work arrangement based on Target’s needs. A Hybrid/Flex for Your Day work arrangement means the team member’s core role will need to be performed both onsite at the Target HQ MN location the role is assigned to and virtually, depending upon what your role, team and tasks require for that day. Work duties cannot be performed outside of the country of the primary work location, unless otherwise prescribed by Target. Click here if you are curious to learn more about Minnesota.

Benefits Eligibility

Please paste this url into your preferred browser to learn about benefits eligibility for this role: https://tgt.biz/BenefitsForYou_D

Americans with Disabilities Act (ADA)

In compliance with state and federal laws, Target will make reasonable accommodations for applicants with disabilities. If a reasonable accommodation is needed to participate in the job application or interview process, please reach out to candidate.accommodations@HRHelp.Target.com.