Automated builds with Travis

All developers should know the value in unit testing and automated builds.  They ensure quality is built and maintained in your software products.  You don’t want to have you interns come in and add some cool functionality into your product and deploy a broken product because they didn’t understand how their changes impacted existing functionality.  In my last post in this series I went over how to create unit tests for a node.js application.  Unit tests by themselves are great, but you (and your intern) need to execute them for there to be any value in having them.  Chances are that intern doesn’t know about unit testing and won’t know they should, or even how to, run the unit tests before committing their changes.  Fortunately there are tools that automate testing and reporting on commits.  Travis is one such tool, and it is free to use.  Setting it up for my node application was stupid easy.  There were three steps, first I needed to define a “default” gulp task like so:

 

gulp.task(‘default’, [‘run-tests’]);

 

This step is necessary because I only defined the “run-test” task in gulp when setting up the unit tests, Travis by default runs the “default” gulp task, so it needs to exist.  Second you need to create a “.travis.yml” file to define how Travis should run.
travis1
Third, you need to log in to Travis with your github account and select the repository to test.  That is it.  You are then running and testing automatically.  If all is setup up right you will see a screen like this shortly after pushing your changes to github, complete with your “build passing” badge.  Awesome!
travis2

Excel PowerPivot: The poor man’s data warehouse

Recently, one of my employees was working to automate an excel based report.  The old school was to Excel automation was involving heavy use of macros and VBA.  This would execute queries or copy data around between sheets spread out on network shares and accomplish some data task.  From my experience this was very brittle and prone to failures.  This time around, I told by employee to use Power Pivot to accomplish the automation.  There is no VBA and only SQL statements that get executed.  The visualization is accomplished by using Pivot Tables and Pivot Charts.  Power Pivot is awesome in that it can pull data from many different types of data sources, SQL, Oracle, SSRS, SSAS, MySQL and Excel Sheets.   The trick is that in order to make your data model work well for reporting you have to think a little like a data warehouse architect.  THis will enable a power user to create self refreshing data sets that can be analyzed in Pivot Tables and Pivot Charts.  Let me walk you through a simple Power Pivot model that I created to track some of my personal fitness goals and health metrics.
Let’s start with the all important time table.  In Data Warehousing, almost all measurable data ties to a point in time, and we model that with a Time Dimension.  In my example, I have a sheet with one column a date and time.  I increment each row by one hour and copied it down about 10,000 times.  You then click “Add to Data Model” and you created a Power Pivot Table.  To make this more useful and interesting for slicing data you need to add some columns.  For Example, add the day of week with this expression: FORMAT([DateTime], “ddd”).  Finally, create a key column.  I learned this trick from a former colleague, you can create a easy to read key for dates and times with a little addition and multiplication.  You could easily exclude that or expand in out to minutes or seconds even: (YEAR ([DateTime]) *10000) + (Month ([DateTime]) * 100) + DAY ([DateTime]).
Adding a Sheet to Power Pivot
add to pivot
Adding Columns
adding columns
Once you have your time situated, you are ready to start creating some tables with data to measure, fact tables in the Data Warehouse lingo.  I wanted to start with three measures: calories, exercise, and weight.  All of these tables started out very simple, there was a column for the date and column for the value.  In the Power Pivot designer I would then repeat the creation of the Date Key on each table.  Final step is to create an association between the fact tables and the time dimension using the calculated DateKey column in Power Pivot; the easiest way to do that is by clicking and dragging in the Power Pivot designer.  Once that is complete you are ready to start adding Pivot Tables or Pivot Charts in your spread sheet.
A Pivot Chart
pivot charts
The Completed Model
model