Home » Archives for Mark Needham

Author Archives: Mark Needham

jq: Cannot iterate over number/string and number cannot be added

In my continued parsing of meetup.com’s JSON API I wanted to extract some information from the following JSON file: $ head -n40 data/members/18313232.json [ { "status": "active", "city": "London", "name": ". .", "other_services": {}, "country": "gb", "topics": [], "lon": -0.13, "joined": 1438866605000, "id": 92951932, "state": "17", "link": "http://www.meetup.com/members/92951932", "photo": { "thumb_link": "http://photos1.meetupstatic.com/photos/member/8/d/6/b/thumb_250896203.jpeg", "photo_id": 250896203, "highres_link": "http://photos1.meetupstatic.com/photos/member/8/d/6/b/highres_250896203.jpeg", "photo_link": "http://photos1.meetupstatic.com/photos/member/8/d/6/b/member_250896203.jpeg" }, "lat": ...

Read More »

jq: Filtering missing keys

I’ve been playing around with the meetup.com API again over the last few days and having saved a set of events to disk I wanted to extract the venues using jq.                                                                                                                                                        This is what a single event record looks like: $ jq -r ".[0]" data/events/0.json { "status": "past", "rating": { "count": 1, "average": 1 }, "utc_offset": 3600000, "event_url": "http://www.meetup.com/londonweb/events/3261890/", ...

Read More »

jq error – Cannot iterate over null (null)

I’ve been playing around with the jq library again over the past couple of days to convert the JSON from the Stack Overflow API into CSV and found myself needing to deal with an optional field. I’ve downloaded 100 or so questions and stored them as an array in a JSON array like so: $ head -n 100 so.json [ ...

Read More »

Deliberate Practice: Building confidence vs practicing

A few weeks ago I wrote about the learning to cycle dependency graph which described some of the skills required to become proficient at riding a bike. While we’ve been practicing various skills/sub skills I’ve often found myself saying the following: if it’s not hard you’re not practicing me, April 2015 i.e. you should find the skill you’re currently practicing ...

Read More »

Incremental/iterative development: Breaking down work

Over the past couple of years I’ve worked on several different applications and one thing they had in common was that they had a huge feature which would take a few months to complete and initially seemed difficult to break down. Since we favoured an incremental/iterative approach to building these features and wanted to add value in short feedback cycles ...

Read More »

Puppet: Keeping the discipline

For the last 5 weeks or so I’ve been working with puppet every day to automate the configuration of various nodes in our stack and my most interesting observation so far is that you really need to keep your discipline when doing this type of work. We can keep that discipline in three main ways when developing modules. Running from ...

Read More »

Pair Programming: Doodling

Another interesting pair programming ‘technique’ which I rediscovered while pairing with Priyank is that of doodling or drawing various parts of the solution when your pair is writing code. I find that this helps to stop my brain wondering off and lets me reflect on what we’re doing from a higher level. As an added bonus it also seems to ...

Read More »

The Five Orders of Ignorance – Phillip G. Armour

While trawling the comments of Dan North’s ‘Deliberate Discovery‘ post I came across an interesting article written by Phillip G. Armour titled ‘The Five Orders of Ignorance‘. The main thing I took from the article is that the author uses the metaphor of software as a ‘knowledge acquisition activity’ for which he then defines five orders of ignorance that we ...

Read More »

Building an API: Test Harness UI

On the project I’ve been working on we’re building an API to be used by other applications in the organisation but when we started none of those applications were ready to integrate with us and therefore drive the API design. Initially we tried driving the API through integration style tests but we realised that taking this approach made it quite ...

Read More »

Performance: Caching per request

A couple of years ago I wrote a post describing an approach my then colleague Christian Blunden used to help improve the performance of an application where you try to do expensive things less or find another way to do them. On the application I’m currently working on we load reference data from an Oracle database into memory based on ...

Read More »