DataCampの学習者データが示す学び方のリアル | Pieter Moors（DataCamp リードデータサイエンティスト）

Generative AI is transforming industries at an unprecedented pace. But as AI changes how you or your team work, one thing is clear. Your skills also need to evolve. At DataCamp, we offer everything you or your team need to adapt and thrive with AI. Whether it's business users looking to get the most out of ChatGPT and Copilot, or developers and data scientists looking to fine-tune models, you can learn the entire AI skills spectrum on DataCamp. Power your AI transformation today. Start learning at datacamp.com. think, well, okay, they might be discouraged and drop off. But for some people, that's really not

the case. They're very persistent. They keep going. As well as teaching people how to work with data, DataCamp also generates a lot of interesting data sets. Today, we're going to discuss some insights that have been discovered by our data team. We'll talk about patterns of DataCamp usage, behavioral levers to help you learn for longer, tips past DataCamp certifications, and insights about the career landscape. Learning to code in our data hasn't become obsolete yet. A term that we all didn't know about three years ago, which is AI engineer, all of a sudden started appearing. It kept rising and rising and rising, and now people who now come onto the platform talk to the assistant about what careers they want, one-on-one data scientist versus AI engineer. I'm joined by Peter Moores, a lead data scientist here at DataCamp. Peter was one of the creators of DataCamp skill assessments and certifications, and now builds data pipelines and analyzes learner data. Let's see what DataCamp learner data has to say.

Hi, Peter. Welcome to the show. Hey, Richie. Very happy to be here. Yeah, great to have you here. So I'm excited because this is a new series we're doing all about exploring some of the data that we find on DataCamp itself. And since you're the lead data scientist here at DataCamp, do you want to talk us through why you decided to analyze some of our internal data? Yeah, so basically we have tons and tons of data on our learners, but we didn't really have a cool way to share it with the people outside of the company. and we thought actually some patterns that we see on learning behavior, on like what is changing in our data over time, how it maps back to reality, to the environment. There's actually some pretty cool insights in there. And that's why we set up like a new program to kind of produce some like insights at the cadence that we try to share like with the world every two weeks or so. And I'm very happy to talk about it, what we've found so far.

Yeah, I'm excited to hear some of the insights. One of the most common questions I get asked in webinars that I run and from speaking to learners is how do I find time to learn? So is there any data on when DataCamp users learn on DataCamp? We can see when people start courses. We can see when people complete them. So we actually have a ton of time on how much time people take for completing courses, but also when they learn. There's actually a very striking pattern to that when people learn on DataCamp, because it's not the same for every type of learner. so we have many enterprise learners people who do learning at data camp because their company gives them the option to do so or their company wants them to learn on new stuff about ai or something like that and like for these people we very typically see a pattern where they learn through their working day and we like i just looked at the data again and we see

people carve out most of the time just before lunch or just after lunch, more or less like that, spread out across the week. So there's not a real pattern of kind of closing out your week with a two-hour learning session or something like that. But they almost never... If you're learning at work, then you can do that joint at work day rather than our weekends. That makes... Before lunch or after lunch are the big times. How about for individuals then, if you're not learning as part of a work program? For individuals, it's not that they don't learn during the day, but we more frequently see them learning in the evenings during the week, and then also during the weekends. About half of the time spent learning by individuals happens in the weekend or in the evenings, whereas only 10% of learning time is devoted in the weekends by our enterprise learners. Okay, yeah, that certainly makes more sense if you're an individual, then maybe you get some time to learn at work, but maybe you have to work, you have to learn in the evenings or maybe you have to work over the weekend. So I can see how that would be a bit more spread out. Are there any kind of extreme cases where you see differing or changes from the normal pattern?

Yes, and one that I found pretty cool myself was also one of the observations that kind of was the one of the first things we did for this insights program was during the World Cup. We were interested to see whether like sometimes you have the feeling that a country pauses like as a whole for like when their national team is playing. In this case, I'm sorry, I'm from Belgium. I know World Cup is equivalent to football or soccer, as some people call it. We were interested in looking at that past summer. And so we thought, okay, let's zoom in on the specific game times of learners in countries where the national teams are playing. and overall we saw a 30% drop in study time compared to the same time a week before, two weeks before when there was nothing special happening and so it just varied a lot

between countries so for some it didn't budge a lot but for most it was between 30-50% to even up to 70% of study time that was dropped so it was pretty cool to see that like as we saw earlier like when people learn at data camp it's in the context of their own lives and when something big happens in the world you see it also in the learning dynamics at data camp as well i can certainly see how data camp as compelling as it is is maybe not as compelling as uh as the the southern world go so yeah uh was there a difference between uh countries then uh were there any countries where you saw a big effect yeah so like the biggest one, if I remember well, was Morocco. Like, they dropped around 70% of their study time. There was one funny observation, which was Belgium, actually. So in our, like, first analysis, we saw that they were the outlier in terms of the fact that they actually added a bit of study time during the game.

The reason why we thought it was funny was that sometimes there's criticism about the national team and, like, maybe people didn't want to watch anymore, blah, blah, blah, blah. But actually there was kind of a confounding factor in the sense that we also have academic groups at DataCamp. So like people doing courses for their study programs And there was a specific university group from a Belgian university doing some crunch time learning on DataCamp during that game And that kind of confounded the results of it. So as soon as we again looked at it more generally, then we saw that trial appear as well. Okay, yeah. I suppose that's like a standard thing with students is that you will do everything at the last possible minute before the deadline. And, you know, maybe that takes priority over even a World Cup game. But the Morocco thing is interesting because Morocco had a really good World Cup. So, yeah, I can certainly see how, like, the whole country was just very excited and maybe it takes priority over data camp. Okay, so before we start talking about differences between learning patterns

from our enterprise users and our individual users, I'm curious as to how much time people spend. So, like, how many times per week people learn, How long do they spend per session? For enterprise learners, we typically see half of the users around that point spends around one day a week, which is kind of in line with their focused learning moment that they carve out. Individuals average more around two days a week because they might be a bit more engaged, they might be a bit more focused to achieve their new goal. But it's not the case that they do a very big chunk of learning typically when they learn. So it's very often like average study times is like between 20 and 30 minutes or so. So it's not that people typically crunch full courses. Like we see them typically do like a lesson, maybe a chapter or something like that. And that's kind of base at which people learn.

Okay, there's interesting that 20 minutes is the standard sort of study length period. I suppose, yeah, after 20 minutes of learning, maybe your brain's full, so you don't necessarily want to do a whole course in one sitting and then not be able to remember it because you've gone through too much. But yeah, learning once or twice a week, that seems like enough to just sort of keep your brain ticking over and your skills gradually sort of staying up to date. I guess related to that, if that's the average learner where they're doing one or two sessions a week, they're doing 20 minutes. Is there like a ramp up from that to like if you are, if you want to become a top learner rather than an average learner? Like are there any sort of techniques to help you learn more or learn better? Like what's very typical of any learning platform, I think it's that cadence is most important. and it often beats big douts of volume that are irregular. So we very, very often see that if you distribute your study across multiple days

and you do this 20, 30-minute sessions or you go maybe up to an hour or something, that makes you come back much more often to the platform rather than you doing it once a week, but then trying to devote two hours of learning time once a week. so yeah i would say make it like something habitual that is that is cadenced very specifically and not too long so that you don't get too tired because in the end yeah learning something new is also a bit tiring okay i would say okay that sounds a bit like um if you want to get fit like you could go to the gym once a week and just wreck yourself but it's probably better to do a little bit of exercise every day and that's going to get you fitter and Exactly. So, okay, so it sounds like learning every day is a good idea. How do you go about building that daily habit of learning? We try to be an engaging learning platform, and so we have things like gamification of features like Streaks that try to encourage you to come back to the platform

and keep your habit going. So the nice thing about Streaks is that you can combine them with practice sessions on a platform. So next to courses, which are a bit bigger and use something new, you can also do practice on the topic of a specific course. And that just takes you about five minutes, if five minutes. And so the combination of keeping your streak while doing practice is a very big lever to make you come back to the platform. and specifically like for keeping streaks we see this like that habit formation is difficult but as soon as you get there you're likely to stick there so the like the going from day one to day two is is very difficult if you if you then go up to day like day seven and even day 14 then it's almost i'm not going to say impossible but it's no longer like a coin flip to lose your streak it's very

likely people are going to want to keep their streak going when they have been learning for 14 consecutive days. For some people, that just makes them come back forever, it seems. We have people who have streaks for almost eight years, and you see it, they just come back every day. If they've done all the courses they want, they just come back to do practice and keep their habit going. that's very cool and two weeks doesn't sound like a a really long time this is a bit like um if you're starting a new diet then you're supposed to do it for two weeks and then you've built the habit and it's easy to continue so certainly uh see how like if you just do it one day it's like oh well maybe it's hard work getting back to day two but yeah just do two weeks and then and then you're there okay does it make a difference if you're learning on desktop or mobile so i know like you can take causes on the app, does that affect your ability to retain the streak? Just to come back to practice, it's very easy to do practice on the app

and to keep your streak going, you don't need to sit at your desk having your desktop with you. But what works best, what we see in the data, to keep your learning going, to not drop out, to keep the habit going is when you try and combine both platforms. So sometimes it's optimal on the mobile app to do the actual learning, to take the courses, because there's videos, there's exercises, that's a bit more difficult. It's very easy to do that on desktop. And then when you do practice to make it a daily habit, when you do that on the app, that's super convenient. And those people, we really see them having a much better retention versus people who do not use the app at all. Okay, yeah, certainly I can see how just doing a few minutes of learning on the app is very convenient. You know, if you're making a cup of coffee

or you're commuting or something like that, then when you don't necessarily have access to a computer, then, you know, you can spend five minutes to do a practice session and that's just keeping your brain ticking over. Okay. Have you seen any other patterns about what helps people learn for longer on DataCamp? It's a bit of a general pattern, I would say, and it's related to the fact how, and I can speak from my own learning on DataCamp, it's how you shape your learning environment or everything we give you to customize the way in which you learn. We have this new AI tutor experience on the platform where rather than having a fixed set of videos and exercises, you actually interact with an AI tutor that kind of shapes the learning towards you as a person. And we see that as very engaging. But like small tweaks, like you can adjust the voice of the person for example to a voice of your own liking That boosts your engagement with the course substantially

But also in the classic experience, and this is more anecdotal rather than backed up with solid data, is whenever you're watching a video, you have a transcript on the side, or you can inspect the slides or something like that. There is very different ways to consume content in also a classic course. that might keep you going more easily because sometimes you don't like to listen to a voice all the time. You just want to go over the slides and then do the exercises or you just want to read what's on the slides. So basically shaping the learning environment towards a learning experience that suits you very well is a powerful way to actually keep learning, I think. That's cool, but just simple tweaks to the user interface just to customize them how you like them. that's going to have a big difference in terms of uh whether you keep learning as you start on the choosing your voice thing i'm actually like um years ago i had this girlfriend who would ignore

the gps whenever we drove anywhere and she's driving crazy i said why are you doing this and it turned out like she really hated the voice of the gps so she would ignore the instructions we had to change we had to change the voice mode on this gps it was fine she would follow the instructions we get to places but like just those little things have a huge difference uh for some people i would also like to talk about certifications that's uh maybe the other thing that uh i usually get a lot of questions from our uh users from so i think uh also this comes up in the data camp sobraddit a lot is like how do you pass certifications or what makes it easier to pass do you have any data on like uh tips you can share for passing the data camp certifications Yeah, maybe it's good to first, like, I don't know how familiar our audience is with our certifications. Should I try to give us a background? So we don't only have like a learning experience at DataGap, but we also have a certification experience.

And that certification experience is actually designed for having like an independent objective, like check of your actual learning on DataGap. So you can complete courses, you can do as much as you want. But if you want to have a valid test of your skills, basically, that's what we design certifications for. And we have very big ones that are focused on roles, like going from data scientist, analyst, to AI engineer. But we also have very small ones, simple ones, like focusing on your SQL skills or your Python skills. so there's like tons of these things for everyone usually the first part of any certification or sometimes the only part is like a timed exam and it's very similar to a data camp assessment that we also have on the platform it's like it's a like a theory of sometimes multiple choice sometimes like coding questions that are very often focused more on like broader knowledge and application of

knowledge, but not necessarily in a real-world environment. That is the second phase of most certifications. And so what we've seen in our data, and I find this very interesting because it's related to the AI tutor that we talked about earlier, is that, of course, when you simply look at assessment scores on DataGamp and people who score high on an assessment that is relevant for the certification, they're more likely to pass the certification. but that's people who already decided to test themselves and so on but so for like the ai tutor one of our hypotheses of why this it's so it's so nice to learn with it is that it like it adapts to your preferences to your like desired speed it will slow down whenever you don't understand something it will give more explanations whenever you want something deeper so it kind of builds that conceptual base better compared to, I would say, in our spec or the normal equivalent, the non-AI tutor ones, the onus is more on you to go deeper and to kind of understand everything better,

whereas the AI tutor might challenge you a bit more on your own understanding. For the AI tutor courses that have relevance for certifications, we see that people who've done those actually have higher chance to pass the timed exams compared to people who didn't have, haven't taken, sorry, the AI tutor courses, but took the non-AI tutor version of it. And so one very concrete suggestion on how to pass these things is, whenever they're available, try to use the AI tutor version. It definitely will have a positive effect on at least getting passed the timed exam in our certification process. That seems like a really straightforward learning change there. Just take AI tutor courses if possible. Actually, when I used to run DataCamp's content quality team, for any given course, the two most common complaints would be, this course is too easy and this course is too hard.

And it's just like getting the difficulty level right for every single user with a traditional format is difficult. But with the AI tutor courses, it does have that level of personalization to match your own, like, any user's ability. So, yeah, if it's teaching you the right speed at the right difficulty level, then, yeah, it gives you a much better chance to pass. Okay. I'd also like to talk a bit about careers. And since everything AI-related is popular right now, does that change the types of jobs that people look for on DataCamp? Yeah, very much so. So one signal, actually two signals. One comes from the users themselves very directly. So we have this AI assistant feature on our platform. And so you can ask it questions simply like, do you have course on X or what should I do next or these kind of things. But very typically when you just are new to the platform,

it will kind of pop up and ask you one of four different things. And for people who are really coming to us to change their careers, which is like there's a bulk of individual learners that really want a career change or they just come straight out of school. They don't know what to do yet. And I want to learn something new. It used to be the case that data scientist was just almost the one thing mentioned, like, this is the role I want to go for, this is what I want to learn, and what do I need to do to become a data scientist? And so over time, in the past two years or so, we've seen that gradually decline. and a term that we, I guess, all didn't know about three years ago, which is AI engineer, all of a sudden started appearing and it kept rising and rising and rising and now people who now come onto the platform talk to the assistant about what careers they want.

It's like one-on-one, like data scientist versus AI engineer. So you can definitely see that what people want to become or what they want to use DataCamp for has changed over time. And we see that, to come back to certifications, we see that in certifications as well. So where we see most growth in certifications is currently in AI engineering certifications on our platform and our data engineering certifications So that the two roles that people are most interested in getting certified for and that we also see in our AI assistants that they mention most to get learning suggestions for. So that shift is for individuals, but for enterprise clients, the change in the type of courses that enterprise users take has been massive. Like, we used to, like, the top courses

used to be intro Python, intro SQL, all those things, but now the top courses are all about AI ethics, AI literacy, AI agents, understanding chat GPT or prompt engineering, all those things, like, enterprise users are really focused on AI upscaling as well. That's absolutely fascinating that AI engineer as a sort of job search term has reached parity with data scientists. I mean, that was where DataCamp started was it was training to help people become data scientists. Obviously, we branched out a lot over the last decade or so. But yeah, the fact that AI engineer is now, I guess, on track to become the top job search term, then that's really interesting. Okay, so I think related to this, maybe one of the biggest fears from a lot of people who are coming to DataGamp is like, is AI going to take my job? Should I still bother to learn to code? Things like this. Do we see any of this fear manifested in the data?

So that's actually where you see an interesting, I'm not going to say paradox, but an interesting trend where although people or individual users frame their goals more in terms of, I want to become an AI engineer, is there any room for doing something else left? all those kinds of things. We don't necessarily see that in the courses they're completing. So they still kind of understand that having a solid basis is still needed even when you interact with AI. Because, well, I guess whatever is being produced by AI, you still need to be comfortable with it somehow. And knowing those basics is still what most individual learners do. before they get into the more technical matter or the more technical details. So learning to code in our data hasn't become obsolete yet. Okay, yeah, I mean, if you want to be an AI engineer,

even if AI can write your code for you, you still need to be able to know how to code yourself. So, yeah, I can see how this is going to be, coding is going to be a skill that's going to be important for a good few years. I'm curious if there are any sticking points that a lot of learners have, because quite often you start off easy and you're writing hello world and then suddenly there's a point where it's like, oh, this is very tricky and a lot of learns get stuck. What do you see in the data? As we all know, I guess, like for the people who have done some programming, most of the basics are very simple, but some of the basics are very confusing. And so something that even now in the age of AI still pops up every week in our Intro to Python course is how indexing works in Python. It's super confusing. People always miss by one or something like that or they don't intuitively understand

how it works. And it's like one of the... Even when AI will always do it right for you, if you need to try and learn it yourself, it's still a major frustration. I can absolutely understand that. Yeah, that's a thing I've fallen over many times, Python indexing syntax, like trying to figure out what you're doing with an array. It's really confusing, especially like the negative indices that are tricky. This is one of those points where like, because I started off in R programming and then moved to Python, I was like, I think R does this one better. All right, so do you have any final advice for us on like how you can learn better on DataCamp? Everything is really about habit formation and it's maybe a bit of a boring advice. But it's kind of true, I think, for if you want to learn something new, you got to carve out time for it. You got to be persistent. I experienced this myself, not in a learning environment, but I tried to start running more, for example,

as a workout. And as soon as it becomes non-negotiable that you go running every other day or so, then it becomes frustrating when you can't do it. So if you can reach the point where learning on data camp or actually not learning on data camp becomes frustrating, that will likely be the point where you want to keep on going, want to keep learning new things, want to kind of consolidate your knowledge. And so, yeah, I think habit formation, being persistent is key. and sometimes we see in our data, like one example is people want to get certified, but they fail the first time, and then you think, well, okay, they might be discouraged and drop off. But for some people, that's really not the case. They're very persistent. They keep going, and there's this, yeah, I noticed there's this one user.

They tried one of our certifications over the course of three years, to obtain it. And then finally, after three years of learning, they finally achieved it. And I think that's not necessarily something you need to aspire to, but it is a nice indication of persistence that if you keep on learning, if you keep on carving out time for it, it's very likely that you'll achieve your goals. In the movies, when people have to learn new things, they just get a training montage and then suddenly they know stuff. And one of the most disappointing things about real life is that it just doesn't work. You just got to put in the time and keep going at it. I was going to say it's like Neo and the Matrix, right? When they upload, or was it Trinity? When they upload new skills to learn and find. That would be a really nice feature of brains if you could just do that. For sure. But that was a very inspiring story from the user who spent three years trying to pass the certification.

I'm curious how many attempts did they have at that to pass? I think it was over like 10 different attempts spread out over time. But if you look into the data, sometimes they want to try it, but they don't even try the exams and then they come back and then they try an exam and then they come back and they try it again or then they pass and they fail. So it's really like, it's almost if this certification attempts was like a feedback booth for a user. This is something I still need to do or this is something I still need to learn or this is something I still am not good enough. That's actually quite nice. It's like you've not failed the exam. You've gotten data on what you still need to learn. Okay. And the persistence of doing that more than 10 times to pass, that's also very inspiring. All right. So, yeah, on that note, thank you for your time, Peter. It was some great insights. It was a pleasure. Thank you.

Thank you.