Senior DevOps Engineer, Applause App Quality, Inc., October 2018 to present

• Led migration of internal services from DC/OS to Kubernetes with minimal developer changes required
• Led migration from Linkerd 1.x to 2.x on Kubernetes
• Led migration of complex Jenkins workflows to GitHub Actions
• Created Kubernetes operator using Ansible Operator SDK for managing MySQL/PostgreSQL databases and users, Datadog monitors/dashboards, and Runscope checks
• Created internal Kubernetes dashboard with SSO auth and proxying to internal webapps configured automatically based on annotations
• Wrote utility in Golang for running process and decrypting environment variables using AWS KMS envelope encryption
• Migrated large hand-built Jenkins setup to Kubernetes with fully automated configuration-as-code setup
• Migrated complex Groovy CI pipeline logic to Python tool to make CI and local developer processes consistent
• Implemented Kong ingress controller and migrated large deployment from older kongfig and PostgreSQL-based setup
• Created automated EKS node replacement process designed for minimal disruption/downtime
• Wrote open-source tool to automate creation of Kubernetes and Kong deploy configs using a simpler YAML-based DSL

Co-Founder & Lead Engineer, Blink Labs, Inc., December 2021 to September 2026

• Created Golang library for communicating with Cardano nodes using the Ouroboros network protocol and reading the blockchain ledger, involving:
  • Extracting protocol details from Haskell code and incomplete specifications
  • Reverse engineering message and data shapes from network packet captures
• Wrote initial implementation of the open-source Dingo Cardano blockchain node in Golang
• Created Cardano blockchain indexer tool with filtering and pluggable output system
• Built a system for tracking domain ownership and serving DNS data from the Cardano blockchain using the custom cDNSd software
• Implemented Golang support for Handshake blockchain network protocol and ledger format
• Contributed code to widely used Golang CBOR library for handling Cardano data shapes that cannot be represented natively, such as maps with bytestring keys
• Delivered grant-funded open-source Cardano tooling through community ecosystem programs
• Contributed to development and testing of next-generation Leios system for increasing throughput and settlement speed of transactions on the Cardano blockchain; implemented early versions of the protocol in Dingo and created automation for the deployment of testing networks

Contract DevOps Engineer, Cloudstruct, LLC, April 2018 to December 2021

• Created system for self-bootstrapping and self-healing Docker Swarm clusters on AWS
• Extensive use of Ansible for automating EC2 deployments, AMI and VMware image building, and management tasks
• Migration of .NET code written in C# from Windows to a Linux/Docker-based deployment
• Provided general DevOps services to a variety of clients

Senior Infrastructure Architect, Cisco, Inc. (formerly Perspica, Inc.), July 2017 to March 2019

• Integrated Perspica's product into the AppDynamics stack
• Automated customized deployment of Kubernetes on AWS using Ansible, kops, Puppet, and Chef
• Created self-bootstrapping "management" AMI for creating new isolated environments with config management, monitoring, and metrics
• Revamped existing image build automation using Packer to properly support AMI and OVF targets
• Designed and implemented automation using Ansible to create "on-premise" application cluster in AWS that uses centralized management and monitoring
• Revamped existing Puppet master, Sensu, Graphite, and other infrastructure services

Contract DevOps Engineer, BitPusher, LLC, February 2017 to August 2017

• Designed and implemented customized OpenStack Ocata deployment using openstack-ansible project and custom playbooks for host network configuration and post-build OpenStack configuration
• Designed and implemented single git repo Terraform setup to manage multiple AWS accounts, multiple environments/stacks, and multiple pieces of stacks, utilizing S3/DynamoDB remote Terraform state

Senior Operations Architect, Turn, Inc., August 2014 to December 2017

• Designed and implemented multi-datacenter Consul cluster with 3500+ nodes
• Created auto-remediation system with Slack integration using Stackstorm, including custom hybrid auth
• Designed and implemented Docker-based replacement for BSD jail-based SFTP system, using Jenkins/Ansible for deployment
• Deployed Ansible across management nodes with custom dynamic inventory script that queries Foreman
• Revamped internal DNS infrastructure, including creating review/deploy pipeline for DNS changes using Jenkins and Ansible
• Designed and built OpenStack clusters using Ceph storage backend
• Created automated build system for Docker images, including creating HA private Docker registry system with access control
• Designed and built HA Mesos/Marathon cluster with Consul and automatic HAProxy configuration
• Migrated from Nagios to Icinga and scaled from single instances to master/slave architecture

Principal Systems Engineer, Box, March 2011 to August 2014

• Played a major role in taking an "organic" environment to a highly scalable enterprise environment
• Rebuilt existing Nagios system to integrate tightly with Puppet for automatic monitoring configuration
• Collaborated closely with engineers to track down various issues causing segfaults and performance degradations in our environment
• Redesigned DNS system to use a series of master and slaves with distribution via git
• Scaled a single-master Puppet system to 20 masters across multiple environments using nginx/unicorn; instituted change management process using git, added validation and automated problem reporting; extended Puppet for non-standard functions using custom facts, parser functions, and providers
• Scaled OpenLDAP across multiple datacenters with two-way replication (limited) using syncrepl
• Led a large-scale migration from CentOS 5.x to CentOS 6.x
• Architected and implemented automated server provisioning using kickstart/cobbler and Puppet
• Designed and implemented a "bastion" server (SSH and squid) to tightly control access to production environment based on LDAP role group membership

Senior System Administrator, Announce Media, August 2009 to March 2011

• Implemented Puppet to manage all of our existing servers and automate the build/configuration of new servers; newly built server could be up and ready for production within 30 minutes
• Implemented an automatic sync between AD and OpenLDAP and transitioned to OpenLDAP for auth
• Managed MySQL replication topology, backup, and administration
• Created an automatic LDAP to MySQL password synchronization process
• Built and maintained multiple Hadoop clusters configured for MapReduce and HBase
• Scaled existing Zabbix setup using MySQL multi-master replication and multiple Zabbix nodes

Senior Linux System Administrator, Broadstripe, December 2008 to August 2009

• Implemented Puppet for configuration management
• Implemented AD authentication on Linux for centralized authentication and access control
• Created DNS cluster for ISP customers and hosting of company-owned domains
• Eliminated wasteful server usage by consolidating functionality and using virtualization

Senior System and Network Administrator, Creative Communications, February 2005 to October 2008

• Designed, implemented, tested, and debugged web-based applications for product ordering/payment, internal inventory management (purchase orders, invoicing, etc.), and internal accounting
• Led migration from hardware terminals to Linux thin clients utilizing a custom telnet-to-LAT gateway to interact with existing MicroVAX systems
• Responsible for Linux server administration including hardware setup, OS installation, server setup/configuration, firewall/router/gateway design/implementation, software upgrades, security updates, and user management

Senior Systems Administrator, Skyline Aeronautics, LLC, October 2001 to August 2005

• Responsible for server administration including hardware setup, OS installation, server setup/configuration, firewall/network design/implementation including DHCP and DNS (bind and dnsmasq), software upgrades, and security updates
• Implemented an e-mail system consisting of SMTP (qmail), POP3, IMAP, and web-based mail for employees
• Designed, built, and maintained a web portal for customers to schedule aircraft and instructors for flight training
• Designed and implemented a Windows NT style domain utilizing Samba 3.0 to take advantage of centralized network logons, roaming profiles, and policies for the publicly accessible computers

Ansible

• Code contributor ("actionable" stdout callback plugin, loop item labels, "piped" SSH transfer method, "openwrt_init" module, various docs fixes, etc.)
• Active community member (answering questions in the #ansible IRC channel on Libera)

Gentoo Linux

Release Engineering Lead (March 2008 to 2012)

• Coordinated the building of Gentoo release media, including stage tarballs, minimal CDs, and Live CD/DVDs
• Implemented the autobuild system (a fresh set of install media for multiple architectures is automatically built, signed, and uploaded to the mirrors on a weekly basis)

Developer (November 2004 to March 2008)

• Developed the Gentoo Linux Installer
• Created numerous enhancements and bug fixes for Gentoo's catalyst and genkernel tools
• Developed the Quickstart utility for doing automated Gentoo installations on multiple architectures (x86, x86_64, hppa, and sparc)