My first job during high school was running the till at a local grocery store near my parents house. The interview process was essentially me walking to the store after school one day with a printed out CV that had ‘I don’t how to do anything, yet’ written on it in fancier words. There was no real job description or any criteria to meet aside from some common sense of ‘yes, I can unpack boxes when stock needs to be put back on the shelves’ and ‘yes, I have basic social skills for talking to customers’.

During my three year tenure at that local grocery store, I spent a majority of my time unpacking boxes and talking to customers.

My next job was at a chain liquor store; the same criteria essentially, but since it was a chain I got benefits like sick leave and PTO. The interview process was a bit more rigorous this time around in the sense that I had to vaguely know the difference between a chardonnay and a sauvignon blanc, but I remember very fondly my first manager (the one who hired me) was asking me the HR mandated questions; one of which was ‘why do you want to work here?’. He chuckled and looked up from his clipboard and said ‘well, that’s a stupid question. The young man is here to get paid, like everyone else, including me’. I worked at this particular shop for 2 years before heading off (back) to university to do a masters.

During the two years of working at the liquor store, I spent a majority of my time serving customers, recommending wine pairings for food, getting yelled at a few times by drunkards and of course, spent a bunch of time unpacking boxes.

Fast-forward several years and you’d find me a proud owner of an extremely expensive piece of paper that had ‘Master of Statistics’ scrawled across it in cursive font; signed by the Vice Chancellor of the institute, no less. I was ready to use my new found skills of programming and analysis in the field - so keenly interested in learning from sagely wizards in the field who had to be significantly more wise than myself since they’d been in the field for so long.

Turns out - I was wrong!

The Wrongness Begins

My first ‘real’ job was working in FinTech. I worked at an organisation that had a keen interest in stopping financial crime, which on paper sounded absolutely sick. “Yeah, dude. Use your programming wizardry to analyse real-time transaction data and develop a machine-learning algorithm that can catch crims trying to steal regular people’s money - it’s gonna be sweet! I promise!” pontificated the squad lead during the interview.

I reviewed the contract I was given at the beginning - but seeing as I was a poor postgraduate student I admit I did skim the details as I was looking to make some money before having to eat crushed pavement as a source of nutrition as opposed to real food from a grocery store which costs money.

Anyway, the general predicate of said contract involved keywords pertaining to programming, statistical analysis, machine learning and prediction. There were some allusions to data visualisation as well, which I also quite enjoyed.

Over the course of six months, I completed the following tasks:

• PowerPoint slides
• Rewriting someone else’s SQL
• Writing a query from scratch because a guy was literally lying about numbers he was producing for a regulatory report to an external governing body
• Rejigging some visualisation dashboards
• Got yelled at by some people from another team

The team itself were a bunch of people I couldn’t really relate to at a human level at all, so not only did I just sit around diddling my thumbs in a stress-inducing environment, I didn't even have any decent colleagues by the end of it. Still, I managed to go six months before quitting.

I also remember telling my team lead about resigning so soon and keenly remember him saying phrases like ‘if you quit now, you’ll ruin your entire career’ and ‘no one will ever hire you with a track record like this’. I am somewhat responsible, so I had already lined up another gig before handing in my resignation.

Spoiler alert: My career wasn’t permanently tarnished as he suggested, although my attitude toward having a job in tech is.

The Wrongness, Redux

The next gig I had was significantly better in many regards. For starters, I knew the people in the team personally and had a rough idea as to what the job would entail.

However, I will mention the interview I had to do.

The Director of Analytics was present, alongside a senior manager and a HR representative. The interview, in my opinion, was not my best work. However, the sweet nepotism that runs the industry saved me from having to show-off too much. Once again, the obligatory ‘we are data driven and are interested in machine learning’ was the main sell of the position. Some things the Director said in the interview were truly baffling. At the time, I assumed I wasn’t smart enough to understand what he was saying; I look back in retrospect and realise he was just an insane person who also didn’t know what he was talking about. As an example, ‘Yes, we decided to aggregate the sentiment in a triangular fashion’ while forming a pyramid shape with his hands.

I managed to secure the role and had a mostly, genuinely good time working here. The team was really nice and we all had lots in common, so the day to day interactions were fantastic. Additionally, I learnt so much about programming from the people here too! Now, to compare the job listing against what I actually did, please refer below:

Description (Data Scientist):

• Machine learning
• Statistical distribution proofing
• Data visualisation

Actual:

• Writing SQL
• Talking to rando’s around the organisation who needed very specific SQL to be run
• Literally creating a relational database in Excel with tables being Excel files in a folder, because the stakeholder refused to let go of the Excel no matter how many times I told them I could do it in a better, easier way
• Made exactly three graphs, one of which ‘oh don’t worry about that, we don’t need it anymore’
• Ran a single t-test across a 2 year period

I don’t want to entirely dunk on it, because the following also happened:

• Linux administration (really cool!)
• Computer hardware retrofitting for a server (probably my favourite task from the job!)
• Wrote a grid search algorithm for Machine Learning feature set optimisation in Python
• Postgres database administration

I worked here for about 2 years before shifting internally to a different team.

The Wrongness, Act III

I soon discovered my penchant for computers was complimenting my career interests, so I decided to shift toward engineering. The organisation had undergone significant restructuring during the tail-end of COVID. Additionally, they had started a new initiative to move all on-prem databases to a cloud solution which opened up a bunch of new positions to shift into.

Although there was a period of time where my position overlapped greatly with the responsibilities of an engineer, I still had to apply internally to secure the role. I even had to do an interview! Let’s talk about it.

The Director of Engineering, the data architect and also a data scientist were the interview panel this time around. All three of them knew me personally and knew exactly what I was capable of (I don’t mean that in an ominous sense, apologies for the phrasing). But, once again let’s compare the position description to what actually happened:

Description for Senior Data Engineer:

• 10+ years of engineering experience
• AWS and Azure infrastructure
• Expert Python programming
• Expert data modelling, schema design and SQL
• Statistical analysis and testing knowledge
• Machine Learning knowledge

What actually happened:

• You guessed it, SQL! But this time it actually (mostly) made sense
• I was never able to actually touch the AWS infrastructure as I wasn’t in the ‘correct’ squad i.e. I never actually wrote a single Lambda function for ingestion across a three year period
• Azure did not appear even once, even in informal conversation
• Got to write some in house Python libraries which was fun
• Rewrote a lot of other people’s SQL
• Decomposed a 2,000 line SQL query because someone in the past thought that was a good idea
• Later decomposed a 1,200 line machine generated SQL string that contained non-standard ASCII white space so it couldn’t compile in Oracle Developer because the PowerBI person who made it ‘didn’t know what to do’ (I mean fair enough, but maybe you should put the keyboard down after about line 200 instead of asking the machine to keep generating more SQL, buddy).

The Wrongful, Fin

So after three years of being SQL man, I decided it was time to finally leave the organisation to explore the world a bit more. Being the fool that I am, I decided to return to FinTech with one massive caveat - I personally knew someone on the team who personally vouched for the ‘goodness’ of working there.

Now, as I mentioned, nepotism is the sweetest nectar and I drank deeply from its cup. I conducted three interviews, of which one was technical. The questions largely alluded to database administration and how table creation works etc. etc. so it sounded promising. There seemed to be a bit more alignment to the position than other places I had worked.

Let’s review (Data Engineer):

• Experience in data warehousing and SQL (5+ years)
• Cloud services for Data-Ops (AWS, GCP etc.)
• Python Programming
• Machine Learning workload/infrastructure needed to do ML work

What happened:

• SQL
• Python programming for checking the state of Database environments i.e. did it run real good or real bad?
• Checking environment of GCP buckets/storage (with Python!)

Wowee! It finally happened! It seemed they aligned the description to what will be happening in the day-to-day (mostly).

Wrong.

I received a cold-call from the platform owner about three weeks into the role. I am informed that the reason I was hired was to do ‘Data-governance, run workshops to upskill other parts of the business and maintain relationships outside of the engineering team to push the usefulness of the platform we’re developing.’

During the three interviews I did, including one with the platform owner there were no direct references to being ‘business partner, data governance guy’. I admit there was one question pertaining to data governance which essentially boiled down to ‘Do you think data governance is a good idea?’ which of course, I said ‘yes, it is’ considering this is a data engineering position.

The Land of Liars

I’m just a guy who’s desperately trying to hail the machine as the machine spirit intended, but I am finding it so incredibly difficult to do so. It seems that no one in this industry is capable of getting to the point of what they actually want, which tells me one of two things:

• They’re lying on purpose
• They genuinely believe what they’re saying is the truth

Both of which are really bad!

I realise some of the points I’ve made were one-off tasks and were frustrating to deal with at the time, but the point I’d like to make is that there’s usually a complete disconnect between what is actually required, what the description says and what an interviewer says. The purpose is lost between the layers I mentioned, when the solution is just to say up front ‘I need you to do the thing, are you capable or interested in doing the thing I need?’.