Senior devops with Ruby/Python coding focus and real-world 24x7 EC2/colo production experience
- 23 years Unix and network administration, 20 years web stack development, 13 years project and team management, 9 years Python, 4 years Ruby.
- Specialist in open-source technologies (LAMP, Linux, Apache, MySQL/PostgreSQL, Python/Perl/Ruby/shell/PHP) and data-center virtualization and automation (Xen/KVM, Puppet, Amazon EC2).
- Customer-focused attitude from professional services and consulting background. Direct, friendly, and respectful communication style. Promotes positive, supportive relationships with all internal and external customers.
- Unix systems administration expert with excellent diagnostic skills. Long-standing daily administration for multiple fully-networked, production-grade hosts (Linux with significant Solaris, BSDs, others). Emphasis on maintainability, self-healing, and automation. Experienced with scaling, clustering, configuration management, and other multiple-system issues.
- Administration focus includes: web (Apache, Tomcat, Python, Rails, PHP), virtualization (Xen, EC2, KVM, Virtualbox), monitoring and automation (Chef, Puppet, Nagios, collectd, Munin, scripting), high availability/scaling, email, Debian/Ubuntu. Also DNS/BIND, MySQL, Subversion, Git, XMPP, TWiki, Tinderbox, Request Tracker, Bugzilla, Mantis, Trac, runit/daemontools/monit.
- Coding and scripting background with Python, Perl, Ruby, PHP, shell. Practical experience with popular web frameworks (Django, Rails, Pylons/TurboGears, Tomcat, Mason).
- Project and team management experience with full life-cycles in teams from 1 to 40 and budgets from $0 to $10 million. Project management skills include interdepartmental communications, requirements analysis, estimation, sprint coordination, and running multiple concurrent projects.
- Full life-cycle sysadmin experience, including development and production environment planning, supporting agile development teams, formal development/staging/QA/production environments, large-scale production environments, network design and management, and desktop support.
Software Engineer (Jan '13 - Present)
- Daily full-stack software development of goodsearch.com using a stack of Ruby on Rails, CoffeeScript, Sass, RSpec, jasmine, PostgreSQL/hstore, sidekiq, RubyMine. All work done using Pivotal-style pairing (two keyboards, one monitor) with daily pair rotation in a team of 8 developers and weekly iterations.
- Manage devops projects and advise development team of operational constraints during planning.
- Lowered hosting operations costs by 50% in 5 months via network and database reimplementations.
Director of Engineering (Feb '12 - Dec '12)
- Lead software development, quality, operations, and analytics functions with team of 17 for web, desktop, and mobile 3D animation and storytelling products. Manage product scope and allocate engineering resources across the product portfolio.
- Supervise and grow iOS capacity from one developer to four and the launch of three iTunes store apps for iPad and iPhone, using native code, PhoneGap, and Unity 3D. Launch new Tellagami business and products.
Director of Infrastructure and Operations (Feb '11 - Feb '12)
- Retain all duties of operations lead per below
- Perform program management duties: track, report, and unblock portfolio of cross-function projects. Perform daily technical project management duties and launch two new product lines targeted at education and professional-user markets.
- Design and implement highly-available job distribution system using evented/async I/O using Twisted and Python. Deploy on Linux controllers and Windows workers with unified codebase.
Operations and Release Manager (Aug '10 - Feb '11)
- Serve as deployment/release manager for all production-facing changes
- Build web and video rendering of 100x operational capacity in 3 months for YouTube-partner launch. Write code to remove scaling bottlenecks by introducing high-availability and load-balancing features. Implement EC2 server array architecture using RightScale-driven scripted server provisioning.
- Sole 24x7 on-call for 110-server operations. Manage software deployments, alerting and incident response, and all server operations.
Operations Manager (Jun '09 - Aug '10)
- Primary operational responsibility for 75 servers, 50TB disk, and 18 redundant MySQL database servers supporting 4 million users and a peak of 750 transactions per second. Responsibilities include daily on-call, application and hardware support, and rollout of new services. [Ubuntu, MySQL, Python]
- Team management of operations function, with cross-department meeting coordination. Project management of a 60-server colo move, a public IP renumbering, and data center contracts.
- Coding of Python, Ruby, and shell tools for systems automation and application metrics. [Nagios, Munin]
- Design and implementation of a Twisted Jabber bot for sysadmin tasks: multi-host app deployment coordination, change notices, inventory lookup. Integrated with Python HTTP-based agent running on all servers. [Twisted, Python, Bottle]
- Led implementation of credit card payment integration and merchant account for SearchBoost product. [Django, Jquery, Recurly, Python]
IT and Service Manager (Jul '05 - Nov '08)
- IT Manager: Ran IT department and help desk servicing 6 organizations, 65 people/desktops, and 25 servers with 2 direct reports. Managed help desk queue and prioritization. Managed vendors, procurement, and departmental budget. Provided email, web, phone, wireless and other office services. Fielded frequent "new project" cycles and performed host/post/webmaster duties across all internal groups. [Windows/Mac OS X/Linux, ticketing, facilities, colo, LAN/WAN, PBX, Apache, blogs/WordPress, Exchange]
- Toolsmith: Supported OSAF development team of 20 and community applications for OSAF and Chandler Project. Created new codebases for administration and public services (server build/migrate/start/stop, public bug submission, web site administration). [Python, Pylons, CherryPy, Subversion, Bugzilla, TWiki, IRC/bots, Tinderbox, Perl]
- Build/Release Master: Served as "release master" for public, open-source Cosmo software releases and associated Chandler Hub service upgrades. Coordinated countdown to "zero bugs", release candidate testing, release notes, and public announcements. Served as "buildmaster" for Cosmo (Java multi-protocol PIM) and Chandler Desktop (Python/C desktop PIM on Windows, Mac, Linux). Managed Subversion tagging and production of various public releases and formats.
- Metrics Manager: Led metrics for OSAF products. Implemented community dashboard and management reports on dashboard.osafoundation.org using new codebase for metrics with data extraction from log files, analysis, storage, and graphing. [Python/Pylons, PostgreSQL, graphing libraries, Google Analytics]
OSAF Volunteer (Dec '08 - Present)
- Served as President and Secretary/Treasurer of the Board of Directors for OSAF.
- Volunteer sysadmin for the OSAF cluster of 12 hosts and service manager for Chandler Hub, a public web service with about 2000 daily users.
Toolsmith, member of Strategic Operations (Nov '03 - Aug '05)
- Built in-house highly-available enterprise monitoring framework for network of 500 heterogenous servers, 10 Oracle databases, and 8 applications totaling 15,000 data streams with 1,100 alerts. [Nagios, Perl, mod_perl, Oracle, Solaris, heartbeat].
- Automation of core enterprise systems, including outbound mail servers and incoming mail requests. Development and maintenance of end-user tools for configuration. [PHP, Perl, cron]
Jared Rhine Business Network, a sole-proprietorship
Various contracts (2000 - 2003)
- All paid gigs: A video transcoding appliance. Web CMS. A data warehouse in SQL Server. A no-hands Linux provisioning CD for an iSCSI appliance. Web performance reports via packet-level timing. Email survey campaign creation and analysis tool. Personalized real-estate listings web app. Product planning for Java 3D data visualization tool. Ran a small application and domain hosting business.
Associate Director of Engineering (Jan '00 - Sep '00)
- Consulted for new service line focused on customer relationship measurement and analysis. Defined product set, roadmap, and initial scope of work.
- Provided Internet expertise to leading school management software company developing $50 million initiative to provide nationwide secure parent/teacher/student extranet. Developed 50-page technical cost model covering planning, development, operations, and maintenance.
Director of Engineering (Oct '98 - Jan '00)
- Profit/Loss (P&L) responsibility for $14M annualized engineering revenue. Tripled department headcount to 115 professionals in six engineering disciplines.
- Accountable for the timely and correct delivery of all software development for ~25 client projects, primarily in Java/J2EE, XML, ATG Dynamo, WebSphere, Oracle, and Solaris. Site functionalities spanned ecommerce, personalization, portal, content, extranets, B2B, customer support. Worked extensively with project teams to troubleshoot projects, mediate interdepartmental issues, and improve engineering execution.
- Directed new business for the engineering group (~60 prospects). Qualified, pitched, closed, and kicked-off new engagements. Responsible for realistic scope and profitability of new project work.
Chief Project Engineer (Sep '97 - Oct '98)
- Started new engineering group composed of technical leads and system architects; grew staff from 3 to 13. Group was responsible for delivery of between 8 and 20 concurrent projects, accounting for ~$7M in annual revenue. Directed an increase in department billable percentage from 25% to 60%.
- Established technical project management standards and methodology. Trained project engineers, engineering teams, and other groups in estimation, scheduling, and risk management.
- Proposed and established Organic's ecommerce system architecture for
first wave of web-based commerce sites.
- Led bi-coastal engineering team of 12 in $1.5M Broadvision/Verity/Mapquest web site relaunch for Home Depot in 90 days.
Engineering Lead (Dec '95 - Sep '97)
- Served as architecture and coding lead for ~20 projects, including McDonald's, Sony Electronics, LucasFilms (Star Wars re-release), US Healthcare/Intelihealth, and AAA. Implementation in CGI, Perl, C/C++, Apache, flat files, MySQL. Supervised small number of direct reports. Mentored junior developers and developed informal coding and process standards.
Software Engineer (Aug '95 - Dec '95)
- Designed and implemented a fully customizable discussion forum software product in Perl for client engagements. Pioneered use of template-driven web pages, defining successful pattern for implementing commercial web sites.
Unix Systems Administrator (Dec '92 - May '95)
- Built and configured first generally accessible Unix server for student body. Sole management of 5 clusters totaling approximately 40 workstations and servers for academic departments. Managed college web, gopher, ftp, and mail services.
- Designed and implemented new college-wide information systems: first web and gopher servers, 600 email distribution lists autogenerated from student and staff rosters, and web, gopher and mailbot accessible archives.
Harvey Mudd College, Claremont CA
- Bachelor of Computer Science. Research focused on networked information discovery and retrieval. Thesis proposed distributed architecture for community-content Internet encyclopedia. Integrated, systems-oriented science curriculum.