New admin screen showing my progress on adding location search
This is the result of weeks of work: a new admin page that says I’m only 38% done.
Booooo.
But. It’s better than it looks. Yesterday I didn’t even know that number.
Weeks ago, it became obvious that Job Garden needed to have some kind of location awareness. I mean, look at Olivier’s board: C4V investments are all pretty international, so there are jobs in Tel Aviv and Paris; in Bristol and New York.
As a job seeker, it would be handy to say “just show me jobs in Europe.”
So I need to extract some kind of verified data, ideally a country and a city.
The thing is that when companies post a new job, they don’t add latitude and longitude. Sometimes, if you’re lucky, there’s a bit of structure (some labeled fields), but often it’s just a few words:
- UK. Well, is that a ISO two character code? I know it means United Kingdom? But the machine, wanting an ISO code, would expect “GB.”
- London. London, UK, or London, Ontario?
- New York. The city or the state? The official name of New York the city is New York City.
- Paris, FR. That I can do, FR is the country code for France. But San Francisco, CA, where “CA” is put in the country field? CA is the country code for Canada.
- Tel-Aviv, Tel Aviv (no hyphen), and “IL — TLV”. These all mean the same thing, and they’re all in job openings posted by the same company.
Looking at these now, it’s easy to figure out what each means.
But there are over 1,000 live roles at Job Garden. So it has to be automated.
And, yes, this is the kind of thing that engineers at Google can do with their eyes closed. But they have a lot more tech that I do, and they also have the luxury of being able to ignore data fields that are hard to understand automatically. I need Job Garden to be more complete than that.
So I’ve been staring at this data trying to think of a neat solution.
For weeks.
It’s a weird way to be blocked, and it happens periodically: there’s some tech or design challenge, and my brain says “hey, there must be a clever way to solve this.”
Often there is. It’ll come to me after a few hours or few days thinking, and I’ll add a bit of code and bingo, problem solved. And it will be neat, and simple, and easy to maintain.
In some other situations, every time I think I’ve found a solution, I realise there’s some case it doesn’t handle and I’m back to square one.
When that happens, I start backing up and backing up and looking for ever more general solutions to broader and broader problems.
That’s what was happening with this location data problem. I’ve been sketching out systems to solve the general geocoding problem and it’s just been too big to hold in my head. So I got stuck.
There’s another approach, which is to do the easy 50%.
Having done the first half, you learn so much that 50% of the rest is now easy.
So you do that. And you learn more, and 50% of the remaining work is now easy.
And so on.
So that was the breakthrough. The “stepping back” I needed to do wasn’t about looking at the technical challenge more broadly, it was to step back and look at my process and realise that it was my way of working that was preventing me from seeing a way forward.
So I did the easy first half. Well actually it wasn’t the first half, it was 38%.
Then I had a look at what was left. “Looking” means building screens like the screenshot above — just a way for me to browse Job Garden and see what’s usually in the database.
I also built views to cluster the types of problem.
For example, are there more problems in verifying the location due to misspelling, absent data, or ambiguities? Looking at only the most common problem, is there perhaps an easy solution?
Once I looked at the actual data I found that what I had been worried about — that is, the cases that had been blocking me for weeks because I couldn’t see how to automate a solution — was only maybe 30 jobs. Everything else looks doable.
30 jobs? I can do that by hand.
Maybe that doesn’t scale to 10,000 jobs, but by that point I will have learnt more.
There’s a phrase I say to myself when I’m hiking and there’s a hairy step to take: trust your boots.
I’ve got to trust that my boots, and my feet, and my legs, and my body, know what they’re doing and they aren’t going to let me down, even if I can’t see exactly what the route is going to be from this side.
So it’s something I should be saying to myself during the working process too. Get stuck in. Start doing the work, and start with the easy stuff. Trust that I’ll learn enough in the thick of it to keep going. Trust your boots.
Location search isn’t released yet, but I can see how far I’ve come and how far I have to go, and that’s this week’s progress.
New admin screen showing my progress on adding location search
This is the result of weeks of work: a new admin page that says I’m only 38% done.
Booooo.
But. It’s better than it looks. Yesterday I didn’t even know that number.
Weeks ago, it became obvious that Job Garden needed to have some kind of location awareness. I mean, look at Olivier’s board: C4V investments are all pretty international, so there are jobs in Tel Aviv and Paris; in Bristol and New York.
As a job seeker, it would be handy to say “just show me jobs in Europe.”
So I need to extract some kind of verified data, ideally a country and a city.
The thing is that when companies post a new job, they don’t add latitude and longitude. Sometimes, if you’re lucky, there’s a bit of structure (some labeled fields), but often it’s just a few words:
Looking at these now, it’s easy to figure out what each means.
But there are over 1,000 live roles at Job Garden. So it has to be automated.
And, yes, this is the kind of thing that engineers at Google can do with their eyes closed. But they have a lot more tech that I do, and they also have the luxury of being able to ignore data fields that are hard to understand automatically. I need Job Garden to be more complete than that.
So I’ve been staring at this data trying to think of a neat solution.
For weeks.
It’s a weird way to be blocked, and it happens periodically: there’s some tech or design challenge, and my brain says “hey, there must be a clever way to solve this.”
Often there is. It’ll come to me after a few hours or few days thinking, and I’ll add a bit of code and bingo, problem solved. And it will be neat, and simple, and easy to maintain.
In some other situations, every time I think I’ve found a solution, I realise there’s some case it doesn’t handle and I’m back to square one.
When that happens, I start backing up and backing up and looking for ever more general solutions to broader and broader problems.
That’s what was happening with this location data problem. I’ve been sketching out systems to solve the general geocoding problem and it’s just been too big to hold in my head. So I got stuck.
There’s another approach, which is to do the easy 50%.
Having done the first half, you learn so much that 50% of the rest is now easy.
So you do that. And you learn more, and 50% of the remaining work is now easy.
And so on.
So that was the breakthrough. The “stepping back” I needed to do wasn’t about looking at the technical challenge more broadly, it was to step back and look at my process and realise that it was my way of working that was preventing me from seeing a way forward.
So I did the easy first half. Well actually it wasn’t the first half, it was 38%.
Then I had a look at what was left. “Looking” means building screens like the screenshot above — just a way for me to browse Job Garden and see what’s usually in the database.
I also built views to cluster the types of problem.
For example, are there more problems in verifying the location due to misspelling, absent data, or ambiguities? Looking at only the most common problem, is there perhaps an easy solution?
Once I looked at the actual data I found that what I had been worried about — that is, the cases that had been blocking me for weeks because I couldn’t see how to automate a solution — was only maybe 30 jobs. Everything else looks doable.
30 jobs? I can do that by hand.
Maybe that doesn’t scale to 10,000 jobs, but by that point I will have learnt more.
There’s a phrase I say to myself when I’m hiking and there’s a hairy step to take: trust your boots.
I’ve got to trust that my boots, and my feet, and my legs, and my body, know what they’re doing and they aren’t going to let me down, even if I can’t see exactly what the route is going to be from this side.
So it’s something I should be saying to myself during the working process too. Get stuck in. Start doing the work, and start with the easy stuff. Trust that I’ll learn enough in the thick of it to keep going. Trust your boots.
Location search isn’t released yet, but I can see how far I’ve come and how far I have to go, and that’s this week’s progress.