5 Takeaways from a mid-level Machine learning Engineer

 The tips that you can learn from this article are so helpful whether you’re at the same career level.

Agenda

• Introduction 
• Don’t be afraid to take risks
• Make it simple 
• Business is involved 
• ML Design Patterns are truth
• Engineering thinking
• Conclusion

Introduction

Foremost what inspired me to do this post is my career situation right now, I’ve been working in a Machine learning area for almost 2/3 years and I’m on the most important project that I’ve ever worked on, I would like to share my experiences and my thoughts about Machine learning engineer role, a lot of people doesn’t know well what ML Engineer role do, but this post is more focused to people which has a certainly mature around machine learning and engineering, Therefore I create a list with a top 5 main takeaways I’ve got working like an ML Engineer from there on out, the thoughts I’ll share can be helpful to another tech careers as well, it isn’t limited to only machine learning engineer.

1 – Don’t be afraid to take risks

Like I said earlier I’m working on the most important project in my career right now, I can’t say what is a project cause is a confidential product we’ve developed, but we all know in MLOps we have got a shortage lack of tools to ensure the best for a machine learning system end-to-end, aspects like data-version, CI/CD focused to ML, deploy huge massive models, monitoring and so on, in the early like a junior ML engineer I avoid to take a risk in some implementations, the big part was because I’ve not a knowledge enough to do, nowadays as a mid-level engineer an I need to take a risks in some implementations same being an uncertainty field yet.

A great example was when I got a task to create an observability feature using Terraform on AWS (I’ve never done it on now) focused on ML, in the time I thought “oh how I can up this feature focused to ML ?” we’ve not had a specific tool to do this work yet, I was challenged to push the boundaries of solutions and make new stuff to solve this problem, I couldn’t understand it like a junior, now as a mid-level, this makes more sense, you’re an engineer not matter if is in software, DevOps, etc you need to foster solutions and take risks for them, always being accountable for this one. 

I was afraid cause it hasn’t solution-focused to machine learning in the industry yet around observability, this can be applied to other scenarios, observability was an example, same with a big lack of tools to solve ML Engineering problems, you’re choice and way is made from scratch, the learning comes with stumbles, not afraid and go on you can achieve the goal.

2- Make it simple 

Following the first takeaway comes to make all work in a simple way possible, like an ML engineer we’ve got a lot of possibilities to create a solution for a Machine learning system. For example, what cloud provider can we use? What is the best repository to use? What will be our feature store? and questions so on, do you agree we have a lot of ways to solve the same problem? ML engineers need to think about all the pipelines and the grainy piece of puzzles.

We need to remember to exist a tenue line if needed and over-engineering, in software engineering the term “over-engineering” refer to in a nutshell excessive design to solve a simple task for ML Engineering the same is valid, “oh what about using a Kubernetes to deploy a unique model” but make sense use Kubernetes only to keep a unique model ? or “what about to up a distributed cluster of GPUs to train a deep learning model” do you really need a deep learning model or a more simple algorithm to solve a problem? you need to be clean in your mind about a solution whose is not necessary to solve this task, how much complexity in your solution more difficult will be to keep it up.

Starts simple and scale from there, you don’t need to scale since the first day of the project, the best solutions are those which solve the problem and are simple, I’ve learned to reduce a complexity solution map to a simple solution map. What do you get with this ? the benefits are many, less time to fix productions errors, clear each other in the team about the solution, easy to hands-up a new feature and many others, my tip is to start simple, keep simple and try scale it simple, your work as an ML Engineer will be easier than ever.

3 – Business is involved

Why this title ?, before I move to ML Engineer I was a Data Scientist a did several models to solve a wide range of business problems, recommendation systems, fraud detection, summarize texts, etc, back then I don’t know about the ML Engineer role, in the first job I did all said before, but this key point was I wasn’t communicative people and like a Data scientist is virtually a skill be a communicative and show the results to stakeholders, PMs and anyone else wanted to know about it, for me it was a problem, I hated to speak with business peoples cause I would rather to code and spend my time in a technical skill than speak in meetings with business peoples, in my opinion, it hasn’t advantage.

well, with a brief introduction about my old role as a Data scientist, I’ve got a chance to move to the ML Engineer role, I’m not switching cause I needed to speak with stakeholders but ML Engineering makes sense for me, I always loved engineering and machine learning, it was a perfect combination one another, but I heard about ML Engineers not need to speak with stakeholders because they are concerned to architecture and deploy, not need to explain models results, metrics, business impact, all this stayed behind, this is a complete mistake.

As an ML Engineer your relationship with business people don’t not extinct, but you’re more concerned to ensure a good ML platform or architecture to a Data Scientist, but business people are involved in this subject, when I heard anyone saying ML engineers needed just code and technical thinking, they were all wrong and I learned which is the impact to PMs are involved with engineering as well, be part of an engineer to stay update with business decisions, you’re not always in charge to technical challenges, is practically impossible to stay far on business and develop a good well-defined so.

4- ML Design Patterns are truth 

what about design patterns turned to machine learning, back time ago some machine learning experts released a book called “Machine Learning Design Patterns” which includes 30 patterns for data and problem representation, operationalization, repeatability, reproducibility, etc, in software engineering this concept of design patterns are stable in ML engineering not yet, I take a stale of how much impact practice this has in a project or product. 

Certainly few peoples know about some of ML design patterns which are more used in machine learning, in my experience when I know some of them and put them into practice, I got a well-defined vision of each part of the project, I knew what is the most risky part of the code, definitely, I was another Engineer before to know and practice ML design patterns.

However, like MLOps practices ML design patterns are equal, depending on your team culture to adopt a practice to use design patterns or not, isn’t a unique decision of you must do but all the team, to finally this topic I’ll share three ML design patterns which most impact my work.

• Embeddings patterns (structure, size of feature vector, loading)
• Stateless Serving Function 
• Feature Store  

5 – Engineering thinking

I was Data Scientist before moving to the ML Engineer role, I’m graduated in computer science (database systems) when I landed my first job in the machine learning area I spent a lot of time studying statistics, probability, linear algebra, all this to achieve a role like Data scientist, in general, I changed my focus into math and forgetting about engineering, for me everything it’s ok, but some things changing in my mind, others experiences convince me about engineering and math applied to machine learning. 

The first one, Engineering is more valuable for an ML Engineer than a math side, you can say, obviously, but I did not see it’s very clear in the beginning, same being machine learning most peoples focus on ML-like the main topic to studied to achieve an ML Engineer role, I’m here to disagree with that, now I realized how much important is to have engineering skills, engineer role is focused to build stuff not craft a machine learning algorithm, is essential to know what is a unit test, deploy, cloud architecture, bash, and more specific practices, when I knew this I could do more productive work.

Conclusion

These were some of my takeaways amongst 2-3 years working as a mid-level Machine learning engineer, I hope this post was helpful to anyone who thinks to pursue a career in ML Engineering, I was inspired to do this post to share with the community my feelings and thoughts about this area, right now ML Engineering is in beginning, we’ve got a lot of things to be a reality yet, therefore this area rising new tools, concepts were born and ML Engineering goes gain maturity and more clarity, thank for reading at the end, see you in the next post.