Site Reliability Engineer (Infrastructure, Virtualization, & Compute)

About the role

SpaceX was founded under the belief that a future where humanity is out exploring the stars is fundamentally more exciting than one where we are not. Today SpaceX is actively developing the technologies to make this possible, with the ultimate goal of enabling human life on Mars.

SITE RELIABILITY ENGINEER (INFRASTRUCTURE, VIRTUALIZATION, & COMPUTE)

SpaceX is looking for an experienced engineer with deep working knowledge of virtualization, infrastructure and compute. This employee will be a member of the Information Technology Infrastructure team and will provide expertise in virtualization/compute, maintenance, scaling and optimization in support of critical business functions. The ideal candidate will be flexible and flourish in a fast paced and challenging environment. They should be a self-starter, self-motivator and possess ingenuity to excel at this position.

RESPONSIBILITIES:

• Design, install, operate, scale, and optimize enterprise compute, virtualization, and storage platforms, including hypervisor clusters, purpose-built servers, and high-performance storage (NAS, SAN, and object storage).
• Work with SpaceX engineers to gather requirements and to design, deploy, and support resilient, high-performance systems for engineering, production, and mission workloads.
• Perform capacity planning and performance engineering, and implement high-availability, backup, replication, and disaster recovery for virtual machines, datastores, and storage arrays.
• Install, configure, patch, and decommission servers, hypervisor hosts, and storage systems, and contribute to hardware and platform standards.
• Automate provisioning, patching, and day-to-day operations with scripting, templates, and infrastructure as code to reduce toil.
• Document standards and best practices, and partner with network, facilities, security, and application teams on connectivity, power and cooling, access, and workload placement.

BASIC QUALIFICATIONS:

• Bachelor’s degree in Computer Science or a STEM discipline and 1+ years of systems engineering experience; OR 3+ years of systems engineering experience in lieu of a degree.
• Experience deploying and supporting Linux servers in physical and virtualized environments (e.g. VMware via automation).
• Experience with the Linux shell as well as configuring and extending Linux instances (e.g. kernel modules, cgroups, pki, iptables, interfaces).

PREFERRED SKILLS AND EXPERIENCE:

• Experience in creating repeatable, reliable, scalable systems architectures, with high availability, fault tolerance, performance tuning, monitoring, and statistics/metrics collection.
• Experience operating production virtualization platforms such as VMware vSphere/vCenter (clustering, HA, DRS, vMotion, datastore design) and/or Hyper-V or KVM.
• Experience architecting and operating enterprise storage, including protocols and platforms such as NFS, SMB/CIFS, iSCSI, Fibre Channel, NVMe-oF, NetApp, Pure Storage, and object storage.
• Deep expertise generating server and host design specifications (CPU, memory, NIC, HBA, local vs. shared storage) to meet defined performance, availability, and density requirements.
• Experience with array- and hypervisor-based snapshots, replication, and backup integration for virtual machine and storage recovery.
• Proven ability to diagnose compute and storage performance issues (IOPS, latency, queue depth, CPU ready/co-stop, datastore contention, fabric/pathing).
• Experience implementing configuration management, provisioning, and workflow automation via infrastructure as code and CI/CD (e.g. Ansible, AWX/Tower, Puppet, cloud-init, Jenkins).
• Experience with programming and scripting languages such as Python, PowerShell, or Golang to automate day-to-day tasks and integrate with external systems through RESTful APIs.
• Experience implementing proactive alert and monitoring workflows and dashboards for Linux systems and virtualization platforms using Prometheus, Grafana, InfluxDB, or similar technologies.
• Experience supporting and scaling containerized applications in Linux environments.
• Experience using automation frameworks (e.g. Ansible, Terraform) to manage provisioning and post-provisioning lifecycles of infrastructure and Kubernetes installations.

ADDITIONAL REQUIREMENTS:

• Must be willing to work extended hours and weekends as needed.

ITAR REQUIREMENTS:

• To conform to U.S. Government export regulations, applicant must be a (i) U.S. citizen or national, (ii) U.S. lawful, permanent resident (aka green card holder), (iii) Refugee under 8 U.S.C. § 1157, or (iv) Asylee under 8 U.S.C. § 1158, or be eligible to obtain the required authorizations from the U.S. Department of State. Learn more about the ITAR here.  

SpaceX is an Equal Opportunity Employer; employment with SpaceX is governed on the basis of merit, competence and qualifications and will not be influenced in any manner by race, color, religion, gender, national origin/ethnicity, veteran status, disability status, age, sexual orientation, gender identity, marital status, mental or physical disability or any other legally protected status.

Applicants wishing to view a copy of SpaceX’s Affirmative Action Plan for veterans and individuals with disabilities, or applicants requiring reasonable accommodation to the application/interview process should reach out to [email protected].