Otis on Leadership

In Shawshank redemption Otis “Red” goes before the parole board with promises to be better. He tries to play by the rules and repeatedly gets denied. Finally, he has had enough and lets them have it. He is done with their game, they can keep him there forever.  He speaks with the tired wisdom of an old man desperate to speak sense to his younger self, bereft of the hope that a future is possible. He doesn’t care, and now they approve his release.

As amusing as this story is, I’m convinced it speaks to a deeper truth. The road to excellence in leadership doesn’t end with learning what to care about, but it definitely starts with a decision to follow your convictions over learning to do what others in power want.

When I was at DARPA, a PM’s goal was to get to the tech council and get the funding to make our idea happen. In several startups, I’ve marched a similar path to get funding. Each time, I had to navigate a maze of intermediaries, each wanting to hear specific things before I could get to the decision maker. The temptation was always present to win their approval by making my main focus to learn and deliver what they wanted to hear at the expense of my core vision. Keeping this temptation in check always helped in the end and I started to really know that conviction matters more than compliance.

Ok, that’s DARPA and startups. But most of my time has been in government and big companies. Big companies don’t just have intermediaries, they have systems, processes and whole organizations that test our compliance versus conviction trade. Conversations in every company start to change as everyone pays attention to where the winds are going. Conviction is still there, but it becomes bounded by what the boss or bosses emphasize. This is a natural consequence of what leadership means. We set the emphasis of our team and that includes culture and values. Shouldn’t we want our workforce to adopt what we are projecting?

Not at the expense of core individual convictions. To get a flavor of how this can lead us off the high road, a senior leader at a former company made (the fine topic of) female empowerment a core platform of his leadership and bombarded LinkedIn with his progress in this area at the expense of any other vision or message. I took pitches every day from vendors and vendor pitches started to include slides at the beginning that highlighted their commitment to female empowerment. This is how you end up with technology pitches that didn’t emphasize technology. While it was great that the boss was speaking his conviction, it was sad to watch the ecosystem around the company pander and step outside their prime value.

Tech vendor’s sometimes do this even when they focus on technology. When a topic becomes hot, say blockchain or machine learning, you start to hear lots of references to high concept phrases. Do you do X? Oh, yes we know X very well. Do you integrate with Y? oh yes. In my research work I get to bump into real thought leaders and it’s a completely different story. They question my question: “why would you do X?”. They often disagree with what I’m saying and point out my misunderstanding. I like these conversations. I like these people. They have different incentives, but they get my call back.

The best conversations are not banal agreements. Listen to a couple on their first date as they try to please each other. It’s funny watching them try and agree. It’s also a boring conversation.

Then watch the verbal tennis match of two long time friends disagreeing. “No, that’s not the best, this is . . . You’re crazy, this is . . .” In such disagreements there is life and learning and love. They deeply care about each other, but they aren’t focused on pleasing the other person. They have transitioned to something greater.

There just isn’t room for multiple things at the top of your priorities. If you focus on playing the game and optimizing the system to your advantage, you not only hurt your chances of success, but you risk any gains you make leaving you empty and not really adding up to any real change. On the other hand, if you really bottom out your convictions and decide what you really want to do, you have to take on the system. The system will fight you and may beat you down. The collective goals of that system will differ from yours and people will defend their equities in ways that give you headaches, sleepless nights and may even break you.

However, some people, and in the right culture, the right people, will watch. They will know that you care more about the impact of your principles than personal gain. They may not agree with you, but they will respect you. And when you succeed your success will have meaning and will take root. It may grow as others are inspired by your conviction and the truth of your principles.

One of my heroes, John Boyd, said you have to make a fundamental decision to the question: do you want to do something or do you want to be somebody? The magic of this is you really can have both, but you have to pick the right door. Both doors will lead to frustration. The choice to be somebody will make you an expert in what people want to hear and where the system is going. It will feed your ego with each win and teach you how to navigate a system with the right partnerships, the right things to do and the right things to believe. Each year you will risk becoming less and less of the person you once were, even if the organization rewards you.

The choice to do something driven by your convictions alone may pit you against the world. John Boyd never became a general. He never developed executive presence and took on the assignments that would get his name at the top of the promotion lists. He lived in a small apartment and his Facebook and LinkedIn page would have been boring and unnoticed. (Hint: I’m pretty sure he wouldn’t have one.) He did change the art of war and gave us our F-16, A-10, and built the science of aerial combat. He made a real impact on national security, but his lasting impact was on his acolytes who went on to change the Air Force and the DoD. His impact lives on.

The choice can’t be more clear. Integrity, conviction and meaning over advancement and ego. It’s Cincinnatus returning to his farm. It’s  General Marshall telling president Roosevelt that general Eisenhower was the better man to lead D-day. It’s George Washington refusing to be crowned King. It’s Socrates taking the cup of hemlock. It’s Martin Luther walking down the streets of Wittenberg with a piece of paper in his hand that will change the world.

The greatest parable is Solomon’s test for the two mothers who both claimed parentage of the same child: who cares more about the baby? When you truly care, you can’t lose. Without Solomon’s wisdom, the true mother would have suffered greatly. She wouldn’t have her baby to touch, to teach and to watch grow. But she could take great joy in the child’s life. There would be a chance for the truth to break free and make all things right.

Even if the other mother “won”, she would live a lie. It would be a parenthood devoid of meaning, filled with guilt and deception. Not living the lie is what bringing your full self means. It means being fully aware of the trades you will make, who you are and what you stand for.

This is all simple when described here, who wouldn’t choose the greater good when it’s described this way? But this gets complicated in Monday’s staff meeting or in your strategy review. Who will you be when you aren’t focused on the questions above? Every day you make decisions that together comprise a life. If you come to forks in the road and you haven’t been intentional about your principles and practiced applying them, then your path will be filled with extra obstacles to find meaning and you risk ending in a tale of sound and fury, in the end signifying nothing. That’s a heavy price to pay for the trophies you get from a happy system.

Making PDFs editable

I automate everything and PDFs are so easy to fill in automatically. It’s a digital world and we just live in it.

It’s so frustrating to get a “protected form” that doesn’t allow for appending documents, automated form completion or even filling out fields. It’s silly because all of these forms are printable. Many years ago, I just created an official form from scratch myself because I couldn’t find it online. Without checking a signature, forms can’t be secure. For example, one could easily write a program that takes a picture and recreates the form. Remember, in the digital world something is either secure from math (crypto) or it isn’t. The security in the transaction is provided by authenticating the user’s email. Even better, let’s move past forms and use the authentication in a website or mobile app.

Let’s do the right thing and move past forms caught in the middle ground of insecure and not usable. I think folks lock forms to prevent changes to the document, don’t do that.

Below is the code to fix this. You will need to use linux or windows subsystem for linux. While I can’t think of an evil use case, use this responsibly. (My goal in doing this is always to fill in data into a form.)

Use a little bash to print with ghostscript. Protip: this isn’t “hacking”, this is just printing.

filename=$1
outfile="${filename%.*}_clean.pdf"
echo $outfile
gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=$outfile -c .setpdfwrite -f $1

New Jersey School Heat Map

This morning before church, I wrote some code to build a contour plot of schools in New Jersey. The “hot” reddish regions have the best schools and the cooler regions don’t.

Tech Stuff: How it’s made

This wasn’t that easy. Since I had all the code from my previous post, it should have been very straightforward to make another contour map. However, to get this map to close, I had four tech challenges to overcome: the GreatSchools API omitted schools when searching from geo-co√∂rdinates, a number of locations didn’t have any schools, and I had to mix python and ruby code in a way that exchanged values.

Fixing GreatSchools Omitted Schools

One of the biggest challenges came from the Great Schools API. While the GS nearby API provides schools within a specified radius of a geographic position, I noticed that schools weren’t showing up within the radius specified. Fortunately, I was familiar with Google’s Directions API, which provides a reverse geocode feature that provides a town name for a given spot. This was far from straightforward since google provides multiple layers of data for each coordinate. On top of that, town names can be locality, postal_town, administrative_area_level_2, administrative_area_level_1. This necessitated the following code from Geoff Boeing:

def parse_city(geocode_data):
    if (not geocode_data is None) and ('address_components' in geocode_data):
        for component in geocode_data['address_components']:
            if 'locality' in component['types']:
                return component['long_name']
            elif 'postal_town' in component['types']:
                return component['long_name']
            elif 'administrative_area_level_2' in component['types']:
                return component['long_name']
            elif 'administrative_area_level_1' in component['types']:
                return component['long_name']
    return None

Fixing locations with no schools

Additionally, many points were not associated with a school. While bodies of water naturally had no school, the Great Schools API failed to report any school for many locations. Since these data didn’t exist, I populated the matrix with NumPy null values or “NaNs”. These showed up as blank regions on the map and wrecked havoc with the contour lines. To fix this, I interpolated in two dimensions using NumPy’s ma.masked_invalid feature followed by SciPy’s interpolate.griddata capability with cubic interpolation. (Wow, Python has a lot of math available on demand.)

Mixed Python and Ruby

The final challenge was connecting to the Great Schools API. I could connect with Python’s default tools, and parse the result with lxml, but that meant building each request. Fortunately, there was a ruby wrapper ready for use, but I had to call it from Python. Python’s subprocess with check_output did the trick, but due to the fickle nature of the API, I had to add a bunch of exception handling.

I’m hoping to make a map like this for northern Virginia soon.

https://gist.github.com/tbbooher/27092a4aa3d6f013e0e07a558c21f5fb

Links

Tax time automation

Happy New Year! Bring in the new year with some code to transform USAA bank statements into a set of transactions for 2016. It is a little tricky because there are some 2015 transactions in the list and USAA puts transactions on two lines.

To use, just save the bank statement pdf as text and run this script. Then you can open the resultant CSV in excel. You will need to have a basic version of ruby installed.

July 4th Home Project: Thermostat Bracket

This post is about how to use design tools to save time and build nice stuff at home using computer controlled machines (CNC). In addition to describing my design process, I’ve also included the helpful references I found along the way.

Our old thermostat was too small for our wall. I could have replaced the drywall, but I needed a starter project to help me understand 3D CNC work with wood. Replacing the drywall would have taken a good bit of time because of the lath behind the old thermostat. The bracket took a long time because I had to learn wood CNC and spent way too long finishing the surface. In the end, this was a great use of a wood CNC machine. It would have been difficult to get the corners right and route out the pocket inside. Additionally, I could prototype and correct parts of my design with the rapid iteration that CNC machines provided.

We have a programmable thermostat with z-wave, the 2gig CT100 Z-Wave Touch Screen Programmable Thermostat. It works perfectly and is easy to control with our Mi Casa Verde VeraLite Home Controller. This gives us the ability to set our temperature from our phones or do nest-like things like learn our patterns and adjust temperature. We can also set up multiple thermostats to regulate temperature throughout the different regions of our house.

In case you are working with the CT100 or the VeraLite, you might find the following links helpful:

Design

I designed the bracket in Fusion 360. I’m still figuring out how to use Fusion, but it is a computer-aided design application for creating 3D digital prototypes including design, visualization and simulation. Fusion 360 is easy to use and it provides the ability to go from design, render, analysis and production in one tool. Most important, it is free for hobbyists.

The design was pretty straightforward. It is a one inch offset with fillets that matched the radius of the CT100. One problem with CNC routing is that I tend to design features that take advantage of the CNC features and this tends to lead to more curves. I just had to get the measurements right. I shouldn’t need to do this, but I used a laser cutter to cut out the frame from a piece of cardboard to check the fit. I’m glad I did, because I hadn’t accounted for some of the curves and the opening was too small. In general, I love using the laser-cutter to prototype designs. The prototype let me see how the final design would look on the wall. This would have been helpful to test different designs. Chrissy and I tend to like 18th-century English and 19th-century neoclassic millwork, but I didn’t put too much thought into this design, partly because I could change it so easily.

Here is the final, dimensioned, design:

Screenshot 2016-07-03 10.16.37

Construction

I found a piece of scrap plywood at TechShop that I cut on the ShopBot buddy.

ShopBot Buddy

To cut the workpiece I used the 1/4″ High Speed Steel Two Flute Downcut. You can see the purchase page here. As this was my first cut, I had to understand the definitions and the different cutter parameters to build the tool in fusion.

For the High Speed Steel Two Flute Downcut I have the parameters are:

  • CED: 1/4
  • CEL: 1
  • SHK: 1/4
  • OAL: 3

Here are some terms that helped me:

CED: CED is abbreviated for cutting edge diameter or the width of the cut the tool should make through the work piece. CED has a tolerance in thousandths of an inch or .xxx decimal places.

CEL: CEL is abbreviated for cutting edge length and is the maximum thickness of the material it can cut. CEL has a tolerance in hundredths of an inch or .xx decimal places.

SHK: SHK is abbreviated for shank diameter and is the nominal size of the shank which should match the collet size of the spindle the tool will be used in. SHK has tolerance in the ten-thousandths of an inch or .xxxx decimal places.

OAL: OAL is abbreviated for overall length and is the total nominal length of the tool from end to end. OAL has a tolerance in hundredths of an inch or .xx decimal places.

HSS: High Speed Steel, typical applications in Non-Abrasive Plastic, Solid Wood & Aluminum where keen edges perform best. High Speed Steel tools perform well in hand routing applications where a tough core is necessary to prevent tool breakage.

Carbide Tipped: Used for a variety of applications in composite woods, hardwoods, abrasive plastics and composites plastics to hard aluminum. Limited by geometry in some applications due to the brazed nature of the tool. Carbide Tipped tools work well in hand routing applications due to the tough HSS core and hard carbide cutting edges.

Solid Carbide: Typically used for widest variety of applications including man-made board, solid wood, abrasive plastics, and some aluminum’s. Solid Carbide does not deflect allowing the tool to be fed at higher feedrates than PCD or straight insert cutters decreasing cycle times. s typically. Solid tools also have major edge keenness advantage thought only possible in HSS until a few years ago.

Chipload: Chipload is simply defined as the thickness of a chip which is formed during the machining of a material. Chipload is critical because if the chip is the proper size, the chip will carry away the heat promoting long tool life. If the chip is too small, the heat is transferred to the cutting tool causing prematurely dulling. Too high of a chipload will cause an unsatisfactory edge finish, or part movement.

The most important reason to understand cutter parameters, is to set the correct feed rates, which is a combination of rpm and cutting speed. In order to get this right, I consulted this reference from ShopBot and read up on end mills in general at makezine. I also was able to incorporate some information from destiny tool that was helpful to verify my settings.

These links also helped:
* hardwood cutting data from Onsrud
* A great video tutorial from ShopBot

After understanding endmills, I had to get everything in the right format for the machine. I found the open sbp reference to be very helpful and the command reference also taught me how to understand the resultant g-code.

I summarized my research below:

Table

Name SB# Onsrud Series Cut Chip Load per leading edge Flutes Feed rate (ips) RPM Max Cut
1/4″ Downcut Carbide End Mill 13507 57-910 1 x D 0.005-0.007 2 3.0-4.2 18,000

You can see the final product here:

20160703_180847

CY2014 Quarter 1 Financial Review

Chrissy and I review our spending on a quarterly basis. Updating every 90 days isn’t too long to correct mistakes and remember purchases, but it also allows for the busy multi-week sprints that life presents us. While we have used every financial management program available, I’ve found the most straightforward and flexible solution is to download historical transactions into Excel where I can assign categories and do the type of analysis you can see below. This works for me because I have complete control. All the other solutions I used (MS Money, Quicken, Mint, GNU Wallet) introduce errors that have required lots of time to fix (or that can’t be fixed), but more importantly they constrain me to their interface and I got used to exporting information into tools that could flexibly answer my questions.

My basic workflow is to download statements from all our bank accounts and credit cards in put them all into one spreadsheet, where I ensure a consistent list of categories. I can do this quickly by filtering and sorting as most of our expenses are cyclical. Once everything is in the right format, I use lots of Excel SUMIF and SUMIFS functions to produce reports.

My purpose of doing a financial review is intended to accomplish the following:

  • Quality check (Are we getting paid the right amounts? Any incorrect expenses?)
  • Spending feedback (Are we overpaying in any categories? Anything we need to reign in?)
  • Tax Production

While my tax production and quality check was very helpful to me, I wanted to share the results of the spend analysis in case my reports might be useful to others.

Spending feedback

In summary, we had a small rise in our overall Grocery and Dining out categories, but the major cost drivers were:

  • Ellie’s 12 cavities were very expensive (no dental insurance)
  • We bought a new espresso machine (major purchase for us)
  • We bought a new car
  • We went crazy on clothes
  • Committed (again) to Army Navy Country Club

spending-by-category

Where are we spending?

This doesn’t have a real effect on our spending, but I thought this was interesting. We don’t have saving/investments in here, this is just “spending”. I treated stuff like insurance, taxes, medical, fees, haircuts, etc as “cost of life” — things I feel we can’t avoid and don’t really have discretion in spending. Some other stuff that might fit this category (power bill) gets lumped into household (as does home maintenance and mortgage). I would love to do some more analysis and compare our spending to this article.

pie

Daily Feedback

The plot below has categories on the Y-axis and days on the bottom. Intensity of color is the spend amount. I used matlab to produce this plot. I like it because the colormap used filters everything in way that comes out like a log scale — and that tells me what is a big deal and what is noise. The interesting dynamic is the frequency/magnitude trade that happens with spending dynamics: medical is in seldom/big chunks while grocery expenses are a constant but smaller expense.

imagesc_daily_spending

You can see that our daily spending has a huge variance: The spending had a standard deviation that was twice our average spending — big purchases had a pronounced effect. I explore four levels of spending: discretionary (dining out), some and limited discretion (haircuts, medical) and non-discretionary (mortgage, tax) at the bottom.

DailySpendingBar

Weekly Feedback

Click on the below to see full size

WeeklySpending

So how much can we control this?

If I break down spending into four categories:

  • Committed — We have to pay it (i.e. Mortgage)
  • Limited Discretion — We can commit extra time to reduce it (i.e. Home and Car Maintenance)
  • Some Discretion — We can make choices to decrease our quality of purchase (i.e. Groceries)
  • Total Discretion — We can do without this if we have to (i.e. Dining Out/New Clothes)

It turns out that a third of our expenses are committed where about a quarter each apply to limited and some discretion. Roughly 20{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} of our expenses are totally discretionary and 70{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} of our expenses could be changed if we had to. The takeaway for me is to focus on eliminating the stuff we pay for but don’t enjoy (fees) and the things that don’t bring joy/reward for their cost.

Excel Sorting and Grouping

I had two tables downloaded from Amazon:

Items

Order Date  Order ID    Title   Category
1/26/14 102-4214073-2201835     Everyday Paleo Family Cookbook
1/13/14 115-8766132-0234619     Awesome Book A
1/13/14 115-8766132-0234619     Awesome Book B

and

Orders

Order Date  Order ID    Subtotal
1/6/14  102-6956821-1091413 $43.20
1/13/14 115-8766130-0234619 $19.42
1/16/14 109-8688911-2954602 $25.86

I’m building our Q1 2014 taxes and needed rows in the following format:

1/13/14 115-8766132-0234619 $22.43 Awesome Book A, Awesome Book B

In order to do this without using SQL, I did the following. If columns B corresponds to Order Id and C corresponds to the item Title, then I put the following formula in column N3

=+IF(B3=B2,N2 & " | " &C2,C3)

and in column O3 a column which might be named: “last before change?”:

=+IF(B3=B4,"", TRUE)

Then I could easily sort out the unwanted values. Done. Still, I would like to better automate this. Any thoughts appreciated.

Practical Cutting Stock Problem

Warning. This post is for an audience of close to zero. My technical friends will find this pretty basic, and my non-technical friends will find this too technical. Welcome to the middle space that I occupy between these two worlds.

I wanted to build this series of retaining walls to replace the overgrown ivy and eroding slope next to my stairs in my front yard.

Retaining Walls

One of the challenges of my current job is to keep my skills sharp as a program manager. I decided to find an automated solution, even when the practical solution was pretty obvious. My problem was basic, I had to make the following cuts:

  • 43 1/8
  • 4′ 7/8
  • 44.5″ x 3

As these were landscape timbers, I knew that I could get them at Home Depot in 8, 10 or 12 foot lengths at $20.57, $29.97, or $37.27. The longer lengths cost more at 2.57, 3, and 3.11 a foot. To minimize my cost, I wanted to buy the least number of pieces and at the shortest lengths possible.

I solved this in 2 minutes in excel, by just guessing and found that I could do one 8 and one 12 or three eights. Three eights would be $62 and and 8,12 would be $58. I’m going to pay the $4 and get pieces I can more easily move, and get 4 more feet to play with.

However, isn’t this a nice knapsack problem for the excel solver? I thought I would give it a try. Google showed me this quick refresher and Wikipedia provides a nice start and the integer program can be formulated as:

$$\min\sum_{i=1}^n c_i x_i$$ $$\text{s.t.}\sum_{i=1}^n a_{ij} x_i \ge q_j, \quad \quad \forall j=1,\dots,m$$ $$x_i \ge 0, \text{integer} $$

where is the number of times order appears in pattern and is the cost (often the waste) of pattern .

However, I wanted to be quick. I didn’t want to have to find all patterns ahead of time. After playing around for a couple minutes, I got this working in excel using a genetic algorithm by forcing each cut to only happen once and minimizing the amount spent.

Excel Solver Example

With these constraints:

Solver Parameters Solver Parameters

While I had a quick solution under my belt, I waited until a long plane flight to cook up something better. First, I skimmed through these papers:

For me, it turns out the key to solving this is to come up with a list of potential patterns and to solve for the pattern combinations in order to get the quantities right. First, I used Matlab to come up with feasible combinations by using the and the very handy allcomb function to generate all possible combinations and removing those that were too long.

{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} here 3 cuts are hard-coded
M = allcomb([0:max_num], [0:max_num], [0:max_num]);

{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} now reduce to the feasible solutions
F = M(M*cuts' < max_length, :);

{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4} remove the trivial
F = F(2:end,:);

First, I implemented this in Excel using their simplex solver.

Improved Solver

With the following solver constraints:

I also implemented an end-to-end solution in Matlab that should be useful for my next project. This way I understand the math behind it and can use my optimizer for any number of projects.

This produces an output like:

board: 1 | quantity: 1 | 2 cuts of 44.5  waste: 7
board: 2 | quantity: 1 | 1 cuts of 44  waste: 52
board: 3 | quantity: 1 | 1 cuts of 44 1 cuts of 44.5  waste: 7.5
board: 4 | quantity: 2 | 1 cuts of 44 1 cuts of 49  waste: 3
----------------------
Total waste: 72.5 inches

And I added some graphics to be able to guide my cuts.

matlab_results

Please share any comments/insights.

“I’m Busy”

As we enter the Christmas/holiday season, I’m asking lots of friends how they are doing and how their year has been. Almost everyone says “it’s busy”. This takes many forms, from “crazy busy” to “absolutely busy” or “scary busy”. Since words often lose their meaning, I’ve thought about a scale we could use to better convey the level of busyness in our lives. I think the best way to delineate levels is by the loss of freedom that occurs at each inflection point.

Continue reading ““I’m Busy””