BayPiggies Write-up: 9/13 SnapLogic Talk

Mike Pittaro, a founder at SnapLogic, spoke on “An Open Source Data Integration Project using Python” at BayPiggies. Slides are available as a PDF. Mike spoke primarily about the issues building the team, infrastructure, and installer for a large Python project.

The SnapLogic product is a data munging and translation application aimed at developers. While the source code is under a GPL license, professional support and service are available for a fee. It has taken about one year to develop. Hiring Python developers was difficult, and eventually good programmers were hired that then learned Python. They are still hiring.

The development infrastructure came together quickly using primarily open source tools written in Python: mailman, subversion, trac, and moinmoin. The team uses both Linux and Windows (using Samba partitions) and both Eclipse/Pydev and vi/emacs. The build process uses buildbot to kick off builds, epydoc and epytext for documentation,figleaf for code coverage, unittest for testing, and (closed source) bitrock for installation. This led to a 11.5 Joel Score.

They learned some rules about coding standards, keeping control of import statements, and wrapping really complex libraries with getattr() tricks. Mostly, they fought with the installation problems of installation of an open source project that requires Python, C, databases, lots of interactions, and also needs to deploy on Windows. They are still fighting with parallelization.

Mike suggested several sources of information. First, keep reading the library reference over and over. He also recommended Python Is Not Java and Code Like a Pythonista.

Thanks again to Mike for a great talk!