Clear Annoying Vim Macros

Recording a macro in vim involves pressing q followed by a letter in visual mode. Next time you press the letter it will run the recording. Handy, but not so great when you accidentally store something in w and then press it when you really meant to save.

The quickest way around this is to press qwq in visual mode. This just records an empty macro for register w.

You can see what’s been recorded by running the :reg command. It’s often possible to work out what it is that you’re accidentally running by looking at the listing.

Macros can be wiped in a more manual way by running :let @w=''. (Replace @w with the register that you corrupted).

Various registers are used by vim to implement the likes of yank so it’s dangerous to arbitrarily clear all the macros, however a simple vimscript function could clear all the alphabetical registers which should be safe.

Clear Annoying Vim Macros

Beginning Unit Tests

The most common thing I hear from people who don’t unit test but would like to is that they don’t know where to start. I’ve documented here the basics of how to write an initial unit test, given some knowledge of Zend Framework and phpunit. I’ve also tried to cover the main points of conflict that developers who have not previously automated their tests tend to run into.

Testing goes from “want” to “have”. So start with a “want”, e.g. “title must be html escaped”. Presumably this “want” will be a description of a new feature or solution to a bug in your software.

Document your want in a test case:

// A PHPUnit subclass with some fixtures to initialise your application
// and make it runnable.
class ProductPageTestCase 
  extends \Zend\Test\PHPUnit\Controller\AbstractHttpControllerTestCase 
{

  public function setUp()
  {
     // ...
  }

  public function testTitleMustBeHtmlEscaped() 
  {
  }
}

Work out how to express your “want” in terms of an assertion. Start at the assertion and work back:

$this->assertContains("<title>" . htmlspecialchars("<") . "</title>", $content);

(I’m using htmlspecialchars because wordpress really loves to ruin escaping in code samples, normally you would write the escaped less-than in the most simple way).

Next work out how to run the page. In the case of a Zend controller test you run:

$this->dispatch("/product-page/1");
$content = $this->getResponse()->getContent();

All that’s happening here is that Zend is simulating the dispatch process that would happen when it receives a real HTTP request.

Work out how you can make the content contain your use case:

$this->database->insert('product', array('product_name' => "<"));

This is the "fixture, run, test" trifecta which you will see in all tests.

Note that what I've written in the example is really an "integration" or "component" test which is testing the full application stack rather than one completely isolated unit. This is usually what you must start with if your application does not currently have any test automation and will give you the most testing value in the fastest way. A unit test of your application classes would need to avoid dependencies as much as possible in order to avoid failing every time a dependency changes. The method of writing the tests is still essentially the same.

The fixture is a point where many get confused. Surely you have to make massive changes to your app to make your database editable from within a test, right? The answer is yes — and that's expected. You are forced into a design by your tests which requires that you make dependencies injectable and mockable. This requires that your application is modular and decoupled which is generally a good thing so the tests are error-checking your design too.

Tools like Zend's service locator are intended to make this kind of design easy, for example a service factory can return an in-memory sqlite for your database and mock transports for your apis when in a test environment. If you're using a modern framework like Zend you may find that even if you're not automating your testing now you are already using the tools you need to do it (but you will need to change the specifics of your implementation to take advantage of that).

In Zend, a test setup fixture might look something like this:

public function setUp() 
{
   // I prefer to use a config which uses my autoloads but skips production
   // credentials and overrides everything to mocked values.  This gives a 
   // short reusable test setup but means your service factories have to do 
   // more work.
   $this->setApplicationConfig(require APP_ROOT . "/config/environments/unit-tests.php");
   $services = $this->getApplicationServiceLocator();
   // You can alter the services manually too:
   $db = new \PDO("sqlite::memory:");
   $db->exec("CREATE TABLE product ( ... )");
   $services->setService("MyAppDb", $db);
}

There are several choices for implementing fixtures. My example simply manipulates the persistence layer directly, which is usually the only choice for existing untested software (at least to begin with). Fixtures can be more readable and further encourage good design if you involve your application's real models (or mocks thereof) rather than bypassing them and using the persistence layer directly.

Generally when you're modifying an existing test case, it should be very obvious how to create a new test method since new methods should look a lot like the existing ones. Structural duplication is expected in a unit test since simplicity — especially avoiding deeply layered software — is more valuable than duplication in test maintenance.

It is also important to be aware that the test will make it hard for you to change the behaviour of that use case or of its fixture. This will make it necessary for you to maintain backwards compatibility features for the sake of your tests. This can be awkward and seems to be a nasty surprise for new developers who are used to breaking compatibility at will. Typically large applications worked on by multiple people need to maintain backwards compatibility anyway so this is another case of the unit tests forcing you into a particular design philosophy which is usually a good idea anyway. Nevertheless this seems to be a huge annoyance among new unit testers so it's worth keeping in mind how your software design needs to change.

Once your test is complete, watch it fail. It should fail in the right way; so in the example above you should expect to see an invalid HTML title produced. This is important because it verifies that your tests are detecting the condition that you intended.

Next implement the feature. Do this in the fastest possible way you can. Don't worry about what sins you have to commit to make the thing happen. Repeatedly run the test until it passes. This is a great way to break out of design worry deadlock and over-engineering — it does not matter how it works so long as it works.

If you find yourself writing lots of print statements at the implementation stage then your test may be too broad. Try abstracting and writing unit tests of the specific part that you are working with. Once again the unit testing is giving you a signal to change the design of your software. In general your tests should be replacing debugging, not adding additional work. Avoiding the debugging process is probably the most important factor for making TDD a fast development process. This can be a hard habit to break and needs some practice. Try implementing a new assertion every time you would have added a print statement.

Once the test passes you can work on cleaning up the code. Remove the sins you committed, clear up and make things pretty. You should not generally need to make major architectural changes at this point; your main job is to remove anything that could be interpreted as duplication, as well as getting rid of future risks like global variables.

If you do this right, you can develop features very quickly without putting any major hurdles into the product for the next time that you have to modify it. Quality trends upwards and your existing code has less impact on the cost of new features. The more you keep the product supple, the more your simple changes become trivial, and your complex changes become achievable.

There is more to testing than this, of course, but the general mantras are:

  • always work from "want" to "have"
  • keep your tests as simple as possible
  • work in small increments
  • don't duplicate code
  • remember the cleanup stage
  • allow the unit tests to steer your design decisions
Beginning Unit Tests

Simulating a High-Latency Network Traffic

Latency is extremely important for web performance since every HTTP connection involves a round trip to the server and there is limited concurrency to avoid this. It can be very useful to see what a high-latency connection will look like and can highlight some problems you were not expecting. Especially useful, it can make the differences between two different implementations more pronounced.

To simulate latency use the tc (traffic control) software from the Linux command line. Adjust the delay as you see fit.

[root@box] # tc qdisc add dev eth0 root netem delay 42ms

This is using the netem module which has several features to configure packet loss, duplication, and latency. (You can find these documented in your systems manual pages of course).

To remove the latency either reboot or run the following:

[root@box] # tc qdisc del dev eth0 root
Simulating a High-Latency Network Traffic

Programming FCB1010 UNO Chip Stomp Boxes for BIAS FX

I really like BIAS FX for headphone playing. It has some limitations and idiosyncrasies but the sound is excellent and the pricing is pretty competitive for what you get.

I decided I would like to control BIAS using a footswitch so I bought a Behringer FCB1010. This worked OK for the expression pedals and I could configure the toggles but the LEDs did not match the toggle state. So I upgraded the FCB1010 with the UNO “unofficial” firmware in order to get its stompbox mode.

Unfortunately people aren’t kidding when they say this is really really obfuscated to configure.

My goal is to configure stomp boxes to work with BIAS FX’s midi learn (which they do not do by default). In this case we need to program a CC to send for each stompbox and pedal and the patch selects are more or less irrelevant.

Global Configuration

I’m assuming the UNO is installed and the memory is initialised. You need to do this in order to enable stompbox.

  • hold “down” while turning on the box

  • turn on LED “9” (this is stompbox mode)

  • turn on LED “8” if you want the stompboxes to be on the upper row. I do not and will assume you keep them on the lower row too, it is relevant in some parts.

  • you might need to turn on LED “5” (disable repeating PCs, I didn’t but I’m using CCs not PCs so I don’t think it applies.)

  • everything else should be turned off by default

  • hold “down” to save and exit (we’re skipping a bunch of screens but that’s fine)

Patch Configuration

To configure stomp box 1.

Configure Patch Stompbox Resets

  • hold down stomp box 1 and the “down” button. It does not seem to matter if you have a patch selected or not but this might just be a display thing.
  • the lights enabled on this screen will tell you which stompboxes will be set/reset/left alone when you switch to the patch. If you set the upper row to be stompboxes then this will be buttons 6-10; otherwise 1-5.

  • hold down each stompbox button to set them to be unchanged.

  • select “up” to move to the next screen

There’s a bunch more stuff you can do here like change the CC of the pedals depending on your stomp and possibly use the set/reset behaviour to make patches out of your stomboxes. You’ll need to experiment with these settings. I haven’t worked out what it does in BIAS yet. I suspect it will not work well as BIAS does not seem to respond to on/off toggles with usually is implemented with with same CC but a 0 or 127 velocity. BIAS appears to merely change toggle state every time a CC arrives, a bit of a shame since the UI does have separate switches for on and off.

Configure Toggle Mode

  • toggle switch “8” on (toggle mode for CC1)
  • toggle switch “9” off (toggle mode for CC2)

  • toggle switch “10” on if you want a momentary switch; off if you want a stompbox.

  • press “up” to continue to the next screen

The toggle modes mean that the switch will send a primary and secondary values separately when enabled. It probably won’t matter for BIAS.

A momentary switch is something that will only stay on while you are holding it down. For some reason this was enabled for all switches by default. There is a way of turning this off globally.

Configure the Stomp Box Value

This should be pretty familiar if you’ve already used the FCB1010 with the standard firmware.

  • everything should be off except “6” which means “CC1”. Hold down switches to disable them. This says what messages will be sent. There is a pre-defined order for sending messages which is in the manual somewhere.
  • press button “6”, it will flash

  • press “up” to configure the controller number

  • use the footswitches or expression pedals to set whatever value you like (I set it to 26)

  • press “up” to configure the primary velocity

  • set it to “127”

  • press “up” to configure the secondary velocity

  • set it to “0”

  • hold “down” to save the changes

Again BIAS does not appear care about the velocities but the way I’ve done this you do need to set them so you may as well set them the standard way. Because of this you will need to manually drag the FXes into the right places to make sure the state matches the LEDs. This is the most messy limitation with this system and means we can’t really be equivalent with a real stomp box.

Conclusions

It may be that my programming can be optimised somewhat. I think the “set/reset” functionality could be promising if only BIAS had a dedicated on/off sensor.

The stomp box (or patches) should be able to be used to change the values of the expression pedals which would make it possible to have a contextual expression pedal for even more flexibility. To be honest single expression pedals are very cheap so perhaps this is overkill.

It would be nice to have some more tactile controls just to change FX plugin values but you still run into the problem where the controls are not in the right state to start with unless there is two way communication between these devices.

It’s sort of annoying that the stompbox signals can’t (as far as I know) be different on a different bank because you’re limited to a mere 5 pedals now.

After all this I would recommend that UNO’s companion editor app is used since everything I’ve done here should be perfectly easy to change using that. Only reason I didn’t use is is that there is a delay in obtaining a license key and I am impatient.

Resources

User guide: http://www.fcb1010.eu/downloads/FCB_UnO_v1_0_4_UserGuide.pdf

Demo of the same process for a G Major 2 (with less explanation about what the stages mean): https://www.youtube.com/watch?v=k0Lp_JuOD1k

Programming FCB1010 UNO Chip Stomp Boxes for BIAS FX

Working Around Merge Conflicts In a Git Rebase

Sometimes you get merge conflicts when rebasing complicated repositories. It doesn’t always happen but it seems to be when existing conflicted merges are resolved. If you’re sure that your rebase doesn’t really conflict with anything in your repository, then you can tell the merge strategy to pick the new change using the following command:

git rebase -s recursive -X theirs master

This just picks the state in the new patch; in other words the diff will reflect the new changes plus any merge resolutions.

It’s still possible to see conflicts when files are deleted and modified in a merge conflict, however you can resolve this with git checkout --theirs FILE ; git add FILE.

I’m not completely sure on why this situation occurs. I think git is simply being safe about how it treats the patches in the history. What we’re doing here is pretending the merge solution never happened and that all commits played well with each other which is not the case in reality. Therefore this command is destroying the history of patches as they actually happened. I’m merging extremely old repositories that have very messy histories anyway so it’s difficult for me to assess the impact of this other than to say that the log looks OK and the files are as they should be.

If all you’re looking for is a nice output of git blame and a bisectable history then this method should be OK for you.

Working Around Merge Conflicts In a Git Rebase

Splitting Subsets of a Git Repository

Sometimes I need to split one or two useful files from a predominantly useless repository. This can be done with subtree merges and subdirectory filters, but if the files are not organised very well then this can prove difficult.

The following quick filter will get you just a few files, providing they don’t move around too much:

git filter-branch --index-filter \ 
  'git rm --cached -qr -- . && \ 
  git reset -q $GIT_COMMIT -- \ 
  FILENAME FILENAME2'

Note that I’m using index-filter rather than tree-filter.

We’re just deleting everything and then resetting the files we want all in one go.

As I wrote in my other post “merging a git repository”, we can move all those files to a subdirectory easily:

git filter-branch --prune-empty \
  --tree-filter 'mkdir -p new-subdir; \
  git ls-tree --name-only $GIT_COMMIT | \
  xargs -I FILES mv FILES new-subdir'

This means we can merge just a tiny part of a repository in very easily.

cd target-repo
git pull --allow-unrelated-histories filtered-repo

Modern versions of git want the --allow-unrelated-histories argument. You’ll need to rebase as well or those commits won’t have any of the stuff your existing repositories had which can prove pretty annoying.

Make sure it’s to your liking by checking out one of the revisions you just imported.

Splitting Subsets of a Git Repository