The Journey to AI as Web Developer
I have been working as a web application developer for the most part of last decade. Back in 2018, I had a colleague who was an avid AI enthusiast. I would be under the spell of his enthusiasm for about a day or two, trying to learn AI. But alas, temporary interest waned, energy dissipated, my mind reverted back firmly to the practical problems of writing reliable, scalable and maintainable web applications.
In 2020, I started thinking about the future of software development. The trending technology then was blockchain which I did not believe in. I turned my attention again to AI. I completed an online course by Andrew Ng and was initiated to the wonderful world of machine learning, but I still had a long way to go.
Fast forward to 2022, the text-to-image application of AI started to create a internet sensation. 6 months later, the appearance of ChatGPT forever changed our expectation of what computers can do. People started to talk about Artificial General Intelligence, and some believed that Skynet from the Terminators could really become a reality within a few decades. Suddenly everything is different for the software engineers. The most important knowledge are no longer API designs or scaling applications, but machine learning engineering.
I was completely unprepared for the disruption. In the last two years I had been reading a lot of books, but mostly books about object oriented design patterns, system architecture and cyber security. Out of the twenty plus books I read, only one book was about machine learning: the “One Hundred Page Machine Learning” book. Suddenly my basic understanding of linear regression, K nearest-neighbour algorithm, neural network forward and backward propagation feel inadequate when the frontiers in the AI field are teaching AI inductive reasoning using RLHF. They have advanced so much from the basic concepts that my knowledge gap looks like the Grand Canyon.
In September 2023, I wrapped up my last job and decided to embark on a journey to teach myself everything about AI. It’s true that the task feels daunting at first, but there are plenty of exciting opportunities. I have to give it a try. My journey so far has not been an easy one, with two major problems:
- The best way to really understand what’s happening in the AI world is to read the published academic papers, but they require a strong academic background in mathematics and computer science. They also require understanding of previous machine learning researches.
- A lot of online tutorial caters to beginners. They show you how to write a short python script to build something with pre-trained models or AI service API. Learning this way is not what I want because I learn nothing about the inner mechanism under the hood. As always the devil is in the details.
The method I employ to tackle the problem is the same for any other problem: make sure I am going in the right direction and take one small step at a time. I can’t afford going back to school so I would have to conquer the missing knowledge blocks by chipping away slowly. Below is a summary of the knowledge I acquired so far and the online resources I found most useful:
- Programming language: Python
If there is one programming language to learn for AI it’s undoubtedly Python. Other languages are trying to compete in this space, including R, Julia, Rust, Mojo, C, and even Javascript, but Python is the king. Luckily I already worked with Python in the past, but if you want to learn it or refresh your memory, I recommend this tutorial
You would probably want to learn about numpy and pandas library too. Reading just enough to get started should be good enough in the beginning.
2. Machine learning concepts
I learned regression, supervised learning and neural network from a course taught by Andrew Ng. The course is great for beginners like myself, with weekly assignments to help me cement understanding and gain deep insights. Find a list of courses by Andrew Ng on Coursea, some of which are completely free:
In addition to the course, I read the one hundred page machine learning book. The material in the book compliments Andrew Ng’s course rather well. I feel the content is just difficult enough to be educational without being prohibitingly difficult.
It was free when I read it, but you may have to pay now:
3. Fun AI projects
Bordom is the enemy of learning. If I keep learning the theories without being able to apply them, there is a real risk of my abandoning the endeavour. In a previous Linkedin post I talked about how I learned just enough to make pet projects, like a proper engineer:
https://www.linkedin.com/feed/update/urn:li:activity:7118027927442636800/
I started out by checking out the tutorial at Lablab.ai. Their tutorials are mostly simple step-by-step instructions to use third party services. You can find similar and probably more idiomatic instructions from the service provider’s official homepage. Though the tutorials lack in the depth I wanted, they allowed me to discover the latest AI trend/capabilities and have some fun.
To get one step closer to AI engineering without a PhD, I highly recommend exploring Hugging Face's website. Hugging Face hosts open source machine learning models, datasets and inference APIs. They also have excellent blogs and documentation of machine learning libraries
I spent a lot of time on Hugging Face's documentation and blog pages, intentionally slowing down my pace to make sure I gained knowledge after reading something (as opposed to rushing through to satisfy my ego). Often I would read a latest blog and google all the unfamiliar terms. I consider their writings a treasure trove.
I read most of the topics in Hugging Face's documentations to get an understanding. Since there are a total of 25 topics there, usually you won’t have enough time to go through all. I highly suggest you focus on Transformers at first.
I also read Hugging Face's blogs daily. It is completely fine if I do not understand everything, as the intended audience of these blogs are machine learning professionals. I exercise judgement and personal preference in determining the amount of time to spend on researching an idea I don’t understand.
4. Online community
I am just recently starting to get involved in the AI enthusiast community. As you can imagine, the learning takes the majority of my time (I also have another non AI project in parallel), but I hope that in the future I would spend more time as part of the community. It’s a great place to explore latest ideas and learn from other people’s experiments. Because of the community’s informal structure, it can be an immersive and fun experience. I often find refreshing opinions that expand my own horizon.
I am following the Reddit machine learning community right now, but there should be plenty of good communities on Discord too.
Though having come a long way since I first started, I still lack many skill sets to build a real world AI application. The knowledge gap can only be closed with grit and time. My plan for the near future is to finish the next two steps:
- Refresh linear algebra knowledge
Even though I consider myself relatively comfortable with linear algebra, I decided to go through a crash course to refresh my memory. One good resource for learning linear algebra is this blog by Pablo Caceres:
Updated Oct 30, 2023: Pablo’s blog contains a lot of good information but unfortunately quite a few obvious errors. I recommend critical examination when using this blog.
2. Learn basic building blocks of machine learning…again
Though I learned machine learning concepts from Andrew Ng’s course and a book, I still don’t feel I completely master the basics. My shortcoming is apparent when I struggled with reading academic papers. I found a great website to learn practically:
This is one of the longest blog I have written so far. Thank you for having the patience to read this far, and I hope my experience is helpful to you.
