FELIPE OLIVEIRA

Search
Close this search box.

FELIPE OLIVEIRA

Search
Close this search box.

5 Takeaways from a mid-level Machine learning Engineer

 

 

 

 The tips that you can learn from this article are so helpful whether you’re at the same career level.

 

Agenda

  • Introduction 
  • Don’t be afraid to take risks
  • Make it simple 
  • Business is involved 
  • ML Design Patterns are truth
  • Engineering thinking
  • Conclusion

Introduction

 

Foremost what inspired me to do this post is my career situation right now, I’ve been working in a Machine learning area for almost 2/3 years and I’m on the most important project that I’ve ever worked on, I would like to share my experiences and my thoughts about Machine learning engineer role, a lot of people doesn’t know well what ML Engineer role do, but this post is more focused to people which has a certainly mature around machine learning and engineering, Therefore I create a list with a top 5 main takeaways I’ve got working like an ML Engineer from there on out, the thoughts I’ll share can be helpful to another tech careers as well, it isn’t limited to only machine learning engineer.

 

 

1 – Don’t be afraid to take risks

 

 

Like I said earlier I’m working on the most important project in my career right now, I can’t say what is a project cause is a confidential product we’ve developed, but we all know in MLOps we have got a shortage lack of tools to ensure the best for a machine learning system end-to-end, aspects like data-version, CI/CD focused to ML, deploy huge massive models, monitoring and so on, in the early like a junior ML engineer I avoid to take a risk in some implementations, the big part was because I’ve not a knowledge enough to do, nowadays as a mid-level engineer an I need to take a risks in some implementations same being an uncertainty field yet.

 

A great example was when I got a task to create an observability feature using Terraform on AWS (I’ve never done it on now) focused on ML, in the time I thought “oh how I can up this feature focused to ML ?” we’ve not had a specific tool to do this work yet, I was challenged to push the boundaries of solutions and make new stuff to solve this problem, I couldn’t understand it like a junior, now as a mid-level, this makes more sense, you’re an engineer not matter if is in software, DevOps, etc you need to foster solutions and take risks for them, always being accountable for this one. 

I was afraid cause it hasn’t solution-focused to machine learning in the industry yet around observability, this can be applied to other scenarios, observability was an example, same with a big lack of tools to solve ML Engineering problems, you’re choice and way is made from scratch, the learning comes with stumbles, not afraid and go on you can achieve the goal.

 

 

 

2- Make it simple 

 

Following the first takeaway comes to make all work in a simple way possible, like an ML engineer we’ve got a lot of possibilities to create a solution for a Machine learning system. For example, what cloud provider can we use? What is the best repository to use? What will be our feature store? and questions so on, do you agree we have a lot of ways to solve the same problem? ML engineers need to think about all the pipelines and the grainy piece of puzzles.

 

We need to remember to exist a tenue line if needed and over-engineering, in software engineering the term “over-engineering” refer to in a nutshell excessive design to solve a simple task for ML Engineering the same is valid, “oh what about using a Kubernetes to deploy a unique model” but make sense use Kubernetes only to keep a unique model ? or “what about to up a distributed cluster of GPUs to train a deep learning model” do you really need a deep learning model or a more simple algorithm to solve a problem? you need to be clean in your mind about a solution whose is not necessary to solve this task, how much complexity in your solution more difficult will be to keep it up.

 

 

 

 

Starts simple and scale from there, you don’t need to scale since the first day of the project, the best solutions are those which solve the problem and are simple, I’ve learned to reduce a complexity solution map to a simple solution map. What do you get with this ? the benefits are many, less time to fix productions errors, clear each other in the team about the solution, easy to hands-up a new feature and many others, my tip is to start simple, keep simple and try scale it simple, your work as an ML Engineer will be easier than ever.

 

 

 

3 – Business is involved

 

Why this title ?, before I move to ML Engineer I was a Data Scientist a did several models to solve a wide range of business problems, recommendation systems, fraud detection, summarize texts, etc, back then I don’t know about the ML Engineer role, in the first job I did all said before, but this key point was I wasn’t communicative people and like a Data scientist is virtually a skill be a communicative and show the results to stakeholders, PMs and anyone else wanted to know about it, for me it was a problem, I hated to speak with business peoples cause I would rather to code and spend my time in a technical skill than speak in meetings with business peoples, in my opinion, it hasn’t advantage.

 

well, with a brief introduction about my old role as a Data scientist, I’ve got a chance to move to the ML Engineer role, I’m not switching cause I needed to speak with stakeholders but ML Engineering makes sense for me, I always loved engineering and machine learning, it was a perfect combination one another, but I heard about ML Engineers not need to speak with stakeholders because they are concerned to architecture and deploy, not need to explain models results, metrics, business impact, all this stayed behind, this is a complete mistake.

 

As an ML Engineer your relationship with business people don’t not extinct, but you’re more concerned to ensure a good ML platform or architecture to a Data Scientist, but business people are involved in this subject, when I heard anyone saying ML engineers needed just code and technical thinking, they were all wrong and I learned which is the impact to PMs are involved with engineering as well, be part of an engineer to stay update with business decisions, you’re not always in charge to technical challenges, is practically impossible to stay far on business and develop a good well-defined so.

 

 

 

4- ML Design Patterns are truth 

 

what about design patterns turned to machine learning, back time ago some machine learning experts released a book called “Machine Learning Design Patterns” which includes 30 patterns for data and problem representation, operationalization, repeatability, reproducibility, etc, in software engineering this concept of design patterns are stable in ML engineering not yet, I take a stale of how much impact practice this has in a project or product. 

 

 

Certainly few peoples know about some of ML design patterns which are more used in machine learning, in my experience when I know some of them and put them into practice, I got a well-defined vision of each part of the project, I knew what is the most risky part of the code, definitely, I was another Engineer before to know and practice ML design patterns.

However, like MLOps practices ML design patterns are equal, depending on your team culture to adopt a practice to use design patterns or not, isn’t a unique decision of you must do but all the team, to finally this topic I’ll share three ML design patterns which most impact my work.

 

  • Embeddings patterns (structure, size of feature vector, loading)
  • Stateless Serving Function 
  • Feature Store  

 

 

 

5 – Engineering thinking

 

I was Data Scientist before moving to the ML Engineer role, I’m graduated in computer science (database systems) when I landed my first job in the machine learning area I spent a lot of time studying statistics, probability, linear algebra, all this to achieve a role like Data scientist, in general, I changed my focus into math and forgetting about engineering, for me everything it’s ok, but some things changing in my mind, others experiences convince me about engineering and math applied to machine learning. 

The first one, Engineering is more valuable for an ML Engineer than a math side, you can say, obviously, but I did not see it’s very clear in the beginning, same being machine learning most peoples focus on ML-like the main topic to studied to achieve an ML Engineer role, I’m here to disagree with that, now I realized how much important is to have engineering skills, engineer role is focused to build stuff not craft a machine learning algorithm, is essential to know what is a unit test, deploy, cloud architecture, bash, and more specific practices, when I knew this I could do more productive work.

 

 

Conclusion

 

These were some of my takeaways amongst 2-3 years working as a mid-level Machine learning engineer, I hope this post was helpful to anyone who thinks to pursue a career in ML Engineering, I was inspired to do this post to share with the community my feelings and thoughts about this area, right now ML Engineering is in beginning, we’ve got a lot of things to be a reality yet, therefore this area rising new tools, concepts were born and ML Engineering goes gain maturity and more clarity, thank for reading at the end, see you in the next post.

 

 

 

 

3,002 Responses

  6. Good V I should definitely pronounce, impressed with your web site. I had no trouble navigating through all the tabs as well as related info ended up being truly simple to do to access. I recently found what I hoped for before you know it in the least. Reasonably unusual. Is likely to appreciate it for those who add forums or something, site theme . a tones way for your customer to communicate. Excellent task..

    Reply

  9. Hello very cool blog!! Man .. Excellent .. Wonderful .. I will bookmark your website and take the feeds also…I’m glad to seek out numerous useful information right here in the post, we’d like develop more strategies on this regard, thank you for sharing.

    Reply

  10. I do agree with all of the ideas you have presented in your post. They’re very convincing and will certainly work. Still, the posts are too short for newbies. Could you please extend them a little from next time? Thanks for the post.

    Reply

  21. It is the best time to make some plans for the future and it’s time to be happy. I have read this post and if I could I want to suggest you few interesting things or advice. Perhaps you can write next articles referring to this article. I wish to read even more things about it!

    Reply

  23. Heya i’m for the first time here. I came across this board and I find It truly useful & it helped me out much. I hope to give something back and help others like you aided me.

    Reply

  24. Pretty section of content. I just stumbled upon your site and in accession capital to assert that I acquire in fact enjoyed account your blog posts. Anyway I will be subscribing to your feeds and even I achievement you access consistently fast.

    Reply

  37. Hiya, I’m really glad I have found this information. Today bloggers publish only about gossips and web and this is actually frustrating. A good website with exciting content, this is what I need. Thank you for keeping this site, I’ll be visiting it. Do you do newsletters? Can not find it.

    Reply

  39. Rattling wonderful information can be found on web site. “The greatest mistake is trying to be more agreeable than you can be.” by Walter Bagehot.

    Reply

  40. I am extremely impressed together with your writing skills as well as with the format to your weblog. Is this a paid theme or did you modify it yourself? Anyway stay up the nice quality writing, it’s rare to see a nice blog like this one these days..

    Reply

  41. Very well written information. It will be helpful to anyone who employess it, including yours truly :). Keep doing what you are doing – can’r wait to read more posts.

    Reply

  43. I truly enjoy looking at on this website , it has excellent blog posts. “Words are, of course, the most powerful drug used by mankind.” by Rudyard Kipling.

    Reply

  44. FitSpresso is a natural weight loss supplement crafted from organic ingredients, offering a safe and side effect-free solution for reducing body weight.

    Reply

  47. For a number of years, Subzerorepair247 has been skillfully fixing a variety of prominent residential ice maker brands throughout the Chicago metropolitan area and neighboring suburbs. Our commitment to quality and affordability has rescued many people from the annoyance of a defective ice maker.

    Reply

  50. Hey! I just wanted to ask if you ever have any problems with hackers? My last blog (wordpress) was hacked and I ended up losing many months of hard work due to no data backup. Do you have any solutions to protect against hackers?

    Reply

  160. Double Glazing Window Repair Tips To Relax Your Daily Lifethe
    One Double Glazing Window Repair Trick That Everyone Should Be Able To window
    repair (Lola)

    Reply

  397. 10kw Multi Fuel Stove Tips To Relax Your Daily Life 10kw Multi Fuel Stove Trick Every Individual Should Be
    Able To 10kw multi fuel stove (Lily)

    Reply

  548. Double Glazing Windows Milton Keynes Tools To Streamline Your Daily LifeThe One Double Glazing Windows Milton Keynes Trick Every Person Should Learn double glazing
    windows milton keynes (Angie)

    Reply

  617. Double Glazed Window Repair Tools To Streamline Your Daily Lifethe One Double Glazed Window Repair Trick That Every Person Should Know window repair [Dong]

    Reply

  690. Hello there! Do you know if they make any plugins to help with SEO?

    I’m trying to get my website to rank for
    some targeted keywords but I’m not seeing very good results.

    If you know of any please share. Cheers! I saw similar blog here:
    Escape rooms

    Reply

  775. I’ve been browsing on-line greater than three hours lately, yet I by no means discovered any interesting article like yours. It’s beautiful worth sufficient for me. Personally, if all web owners and bloggers made just right content as you probably did, the net can be much more useful than ever before!

    Reply

  833. Double Glazing Window Repair Tools To Help You Manage Your Everyday Lifethe
    Only Double Glazing Window Repair Trick That Should Be Used
    By Everyone Know window repair; Elton,

    Reply

  868. Who Repairs Sliding Patio Doors Techniques To Simplify Your Everyday
    Lifethe Only Who Repairs Sliding Patio Doors Trick Every Individual Should Know who repairs sliding patio doors (Natasha)

    Reply

  1089. Your way of telling the whole thing in this paragraph is truly nice, every one can simply understand it, Thanks a lot.

    Feel free to surf to my web blog; next

    Reply

  1214. Double Glazing Installers Near Me Tools To Help You Manage Your
    Daily Life Double Glazing Installers Near Me Trick That Every Person Must Know
    near

    Reply

  1580. Asbestos
    is a hazardous naturally occurring mineral used
    to create insulation and fire resistance. It was used for decades in New York, as well as other
    areas of the country in manufacturing, industrial and commercial
    environments.

    Reply

  1690. Asbestos is a naturally occurring mineral that was previously extensively used
    in construction projects and manufacturing industries.
    Asbestos exposure was frequently associated with serious, and sometimes life-threatening illnesses.

    Reply

  2048. A fascinating discussion is worth comment. I do believe that you should write more about this topic, it may not be a taboo subject but generally folks don’t discuss such issues. To the next! Kind regards!

    Reply

  2078. When I originally left a comment I seem to have clicked the -Notify me when new comments are added- checkbox and from now on each time a comment is added I recieve 4 emails with the same comment. There has to be a way you are able to remove me from that service? Cheers.

    Reply

  2080. Mesothelioma patients in New York deserve the maximum possible compensation. Expert
    asbestos Lawyer
    lawyers, like the ones at Lipsitz Green Scime Cambria, can assist.
    They can negotiate settlements worth more than a million dollars for
    their clients.

    Reply

  2151. Hi there! I could have sworn I’ve visited your blog before but after browsing through a few of the posts I realized it’s new to me. Anyways, I’m definitely pleased I discovered it and I’ll be book-marking it and checking back often.

    Reply

  2205. Greetings, I think your web site may be having web browser compatibility issues. Whenever I look at your site in Safari, it looks fine however, when opening in Internet Explorer, it has some overlapping issues. I merely wanted to give you a quick heads up! Other than that, fantastic blog.

    Reply

  2580. Coffee Machines From Bean To Cup Tools To Improve Your Daily Life Coffee Machines From Bean To Cup Trick That Everyone Should Learn coffee machines from bean to cup, Cherie,

    Reply

  2620. A reputable mesothelioma lawyer will offer a no-cost case evaluation to determine whether you qualify
    for compensation. The best mesothelioma attorneys also operate on a contingent basis.
    This arrangement maximizes compensation for clients.

    Here is my homepage asbestos

    Reply

  2646. Mesothelioma is cancer of the mesothelium which forms a line around certain organs, such as the lungs.
    The most frequent type is pleural msothelioma which is a cancer
    that develops in the lining of the lungs.

    my page :: asbestos

    Reply

  2730. What i do not understood is in fact how you are now not
    actually much more smartly-favored than you may be now. You are so
    intelligent. You already know thus considerably in relation to this subject, produced me in my view believe it
    from a lot of numerous angles. Its like women and
    men don’t seem to be fascinated unless it is something to
    do with Girl gaga! Your individual stuffs great. At all times
    deal with it up!

    My homepage … click here for more

    Reply

  2826. This design is wicked! You most certainly know how to keep a reader amused.
    Between your wit and your videos, I was almost moved
    to start my own blog (well, almost…HaHa!) Great job.

    I really loved what you had to say, and more than that, how you presented it.
    Too cool!

    Here is my blog … additional hints

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Felipe Oliveira

Felipe Oliveira

FELIPE OLIVEIRA

Linkedin Envelope