Congressional Districts in Full JSON Format

The challenge is how to get ERSI shapefiles into a full json format.

I had MATLAB fired up and was going to generate this originally. It turns out a lot of the data are already there. Experience matters *a lot*. It took me two weeks to do this before. This time it took about 5 hours.

I still have all the MATLAB code, along with my custom gis calculations, but this puts everything into the database and is much cleaner.

Removing Images of a certain size

I had a bunch of bogus images in my directory and I was going through and manually deleting them. I wised up a little bit and thought I would write a quick script. While it ran, I thought I would share with you.

I had: 25760 images and probably about 15,000 of them were bunk. I could use identify from ImageMagik to get the data which contained the width and height of the image. All the bogus images were a specific height. By writing a script I was able to put them all into a directory, which I could then review for deletion.

Enjoy

Import Access Data into Excel (via VB and ADO)

I had a lot of data in MySql in a rails application, but I moved to Mongo on a new application and wanted to give the client a nice archive of excel files for each client. It has been a long time since I’ve worked with visual basic, but it came back quickly.

I was amazed how clunky the interface and I wrote some pretty redundant code, but this was my Saturday and I tried to get this out the door quickly. My basic workflow was to load an office trial version on a virtual machine, then install SQL server on windows and set up an ODBC connection string which I used to import all data into access. In access, I just wrote a bunch of sql strings and used access to automate excel via VBA.

https://gist.github.com/tbbooher/2386826

Wind speed calculation

I want to know the impact of wind on endurance.

So we have something like this:

This gives us 8 cases of wind direction:

From this we can infer that going to the center from one of the 8 directions.

So an overall flight path would look like this:

If each grid cell is a distance (s) in size, each time a diagonal transition is made the distance would be (sqrt{2} s). As an aircraft enters each cell, they encounter the wind-star as shown above with the expected value of the wind facing one of the eight possible directions. In general, if (W) is the wind velocity in direction (alpha), from grid direction (g in [1,8]) we have to find the expected value for the wind in direction (cos(alpha_{g,w})), by integrating over all values of (w).

[
V_{g,w} = sum_{1..12} cos(alpha_{g,w_{i}}) dot P_i
]

With these bins:

h3. Wind Directions ( w_{i} )
|Wind Direction|Angle|
|1|0|
|2|30|
|3|60|
|4|90|
|5|120|
|6|150|
|7|180|
|8|210|
|9|240|
|10|270|
|11|300|
|12|330|

h3. Grid Direction (g)
|Grid Direction||
|1|315|
|2|270|
|3|225|
|4|0|
|5|180|
|6|45|
|7|90|
|8|135|

Radar parameter estimation

Here we tackle what a Radar needs to accurately measure a target’s position, size and motion. (This is nothing new, just a placeholder for some of my notes.)

First of all, everything depends on $SNR$, and the parameter estimation happens in the main computer of a Radar. The theoretical rms error of a measurement is of the form:

where \(k\) is a constant between 0.5 and 1.

h3. Range

First, let’s tackle range. They key metric for range is bandwidth, the higher the bandwidth the better the resolution.

Estimation of the range of a target is based upon using A/D sampled measurements of the round trip time to and the target

For time delay measurements , such as range, the value of the constant depends on the shape of the radar pulse’s spectrum and the pulse’s rise time and for a rectangular pulse, whose width is, (T):

or equivalently,

However, pulses always move in trains or if $T_D$ is the dwell time:

from Barton and Ward.

So Range depends on \(SNR\), pulse shape and width, effective bandwidth, number of pulses that can be coherently integrated.

This is explained well by a diagram in the lincoln presentation:

h3. Angle

Now, let’s tackle angle. Angle accuracy depends on antenna size. Simple detection provides coarse location in angle by isolating a target’s location within beamwidth of antenna (and beamwidth is just \(\lambda / D \), where \(D\) is the aperture diameter and \(\lambda \) is the wavelength. But this is very coarse, 1° beam at 100 km extends across 1,745 meters. In order to provide more accurate angle estimation, you have to measure at different beam positions to improve accuracy. You can do this through sequential lobing, conical scan tracking, or monopulse angle estimation.

h4. Conical Scanning

Conical scanning is a technique to keep the beam pointed at the target to improve angle accuracy. It is based on the principle that the radar receiver will get maximum returned signal strength when the target is in the beam centre, so if the beam is pointed directly at the target, when the target moves it will move out of the beam center and the received signal strength will fall. Circuitry designed to monitor any fall off in received signal strength controls a servo motor that steers the aerial to follow the target motion.

Wikipedia has a nice drawing that explains this technique:

!https://upload.wikimedia.org/wikipedia/commons/3/32/Conical_scan.jpg!

h4. Sequential lobing

While conical scan tracking uses the phase of modulation to get the angle error and amplitude modulation to get the beam displacement, sequential lobing uses two slightly separated antenna elements to send the beam slightly to either side of the midline of the antenna, switching between the two to find which one gave the stronger return, thereby indicating which direction the antenna should be moved in order to point directly at the target.

Here, wikipedia provides another good diagram:

!https://upload.wikimedia.org/wikipedia/commons/b/b6/Monopulseprinzip2.gif!

h4. Monopulse

Monopulse radars are similar in general construction to conical scanning systems, but add one more feature. Instead of broadcasting the signal out of the antenna “as is”, they split the beam into parts and then send the two signals out of the antenna in slightly different directions. When the reflected signals are received they are amplified separately and compared to each other, indicating which direction has a stronger return, and thus the general direction of the target relative to the boresight. While monopulse, requires a separate receiver for each channel, it improves performance over conical scan and sequential lobing whose performance degrade with time varying radar returns.

All angle errors are summed up well by the following:

and

SNR, type of measurement technique, antenna illumination distribution, antenna size, and frequency.

h3. Doppler

Lastly, let’s look at Doppler accuracy. Doppler accuracy depends on coherent integration time \(\Delta t \):

Doppler frequency is calculated as:

To accurately measure Doppler, use two closely spaced frequency filters offset from the center frequency of the Doppler filter containing the detection

Doppler accuracy depends on SNR, pulse shape, integration time.

h3. Real world limitations

h4. Receiver noise

# Adds variance to estimates

h4. Radar calibration

# Poor calibration leads to poor estimation

h4. Amplitude fluctuations

# Small effect on monopulse and array solutions

h4. Angle noise (angle scintillations, or target glint)

# Complex target return biases angle estimate

h4. Multipath (low angle tracking)

# Reflection off earth’s surface combines with direct path return

# Can cause biases in angle estimates for all technique

h4. Good References

“Parameter Estimation”:http://www.ll.mit.edu/workshops/education/videocourses/introradar/lecture9/lecture.pdf

“Basic Fundamentals”:http://faculty.nps.edu/jenn/Seminars/RadarFundamentals.pdf

“Parameter Estimation and Tracking”:http://aess.cs.unh.edu/Radar{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}202010{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}20PDFs/Radar{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}202009{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}20A_15{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}20Parameter{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}20Estimation{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}20and{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}20Tracking{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}20Part{aaa01f1184b23bc5204459599a780c2efd1a71f819cd2b338cab4b7a2f8e97d4}201.pdf

Render CSV in Rails

First I read the docs at: rubydoc.info to understand how FasterCSV is now in the ruby standard library.

Then, I set up the following code:

https://gist.github.com/tbbooher/1700200

This gives me a pretty flexible solution to render csv, but I’m really interested to see if this is a good way to go. Your comments are appreciated.

Your key with fingerprint … is not authorized to access heroku with Ubuntu

This error was pretty hard to solve. I emailed heroku support and they recommended:

“https://github.com/ddollar/heroku-accounts”:https://github.com/ddollar/heroku-accounts

bq. For git commands to work git needs to use the ssh key associated with that account. It’s probably choosing a key associated with a different account which is causing these permissions errors. You can use a ~/.ssh/config file to specify which key should be used. Check out this client plugin written by one of our engineers. It helps manage both Heroku credentials and ssh keys for easy switching between accounts. https://github.com/ddollar/heroku-accounts Please let me know if you have more questions.

Their comment did make my future work with multiple accounts easier — it didn’t solve my problem which was that my old fingerprint was still active, and causing trouble.

The solution for me was not to create and provide a new key to heroku. It was to go to the gnome passwords and keys folder and manually delete “My Personal Keys”.

See this comment from “launchpad”:https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/505278 for details.

bq. The culprit is gpg-keyring-daemon. It subverts the normal operation of ssh-agent, mostly just so that it can pop up a pretty box into which you can type the passphrase for an encrypted ssh key. And it paws through your .ssh directory, and automatically adds any keys it finds to your agent. And it won’t let you delete those keys. How do we hate this? Let’s not count the ways — life’s too short.

bq. The failure is compounded because newer ssh clients automatically try all the keys in your ssh-agent when connecting to a host. If there are too many, the server will reject the connection. And since gnome-keyring-daemon has decided for itself how many keys you want your ssh-agent to have, and has autoloaded them, AND WON’T LET YOU DELETE THEM, you’re toast.

bq. What you really want to do is to turn off gpg-keyring-daemon altogether. Go to System –> Preferences –> Startup Applications, and unselect the “SSH Key Agent (Gnome Keyring SSH Agent)” box — you’ll need to scroll down to find it.

bq. You’ll still get an ssh-agent, only now it will behave sanely: no keys autoloaded, you run ssh-add to add them, and if you want to delete keys, you can. Imagine that.

UTAPS: Mark an IDT as complete

UTAPS is terrible software. I wasted a half hour today going through the source code to figure out how to mark an IDT as complete.

I have no idea how folks who don’t understand web codes can navigate this system.

Why not include a button? Really?