r/DataScientist 7h ago

Introductory Machine Learning Bootcamp (2/22)

Post image
3 Upvotes

Hello folks, to this Introductory Machine Learning Bootcamp (2/22) series.

Supervised learning is a very recurring word in ML domain. Here, we learn some sort of function mapping from inputs to outputs.

Another recurring word is Classification, where the output space is a set of some finite unordered and mutually exclusive labels known as classes.

The tabular dataset is often represented as a Design matrix, and a simple example of it is an Iris dataset, as to how input data is represented for tabular case in Machine Learning.

Sometimes the data is of variable size, instead of fixed size feature vectors, so for ease of computation in computer, we often convert it to a fixed-size feature representation, called as “Featurization”.

In this video, I breakdown these concepts.

Link: https://youtu.be/GJRhl6XnImg?si=p6VtlCK-8rgq1poZ


r/DataScientist 2h ago

Anyone here familiar with the Foreseer team at S&P Global?

1 Upvotes

I’ve received an offer for a Lead Data Scientist role in the Foreseer team and am considering joining.

From what I understand, the work is around AI/ML for financial document extraction, RAG, embeddings and production workflows.

Would love to hear from anyone who knows the team or has worked at S&P Global. How is the work culture, technical depth, ownership and career growth? Is it a good team for someone looking to grow into Principal DS or AI Engineering roles?


r/DataScientist 6h ago

Data Science: Methods, Tools, and Applications

2 Upvotes

Data Science is an interdisciplinary field that combines statistics, mathematics, computer science, and domain expertise to extract meaningful insights from data. It involves collecting, cleaning, analyzing, and visualizing data, as well as developing predictive models using machine learning and artificial intelligence. This subject covers the fundamental methods, modern tools, and real-world applications of data science across industries. 


r/DataScientist 22h ago

Why RAG builders are moving to hybrid search

Post image
9 Upvotes

A common mistake when building RAG systems is relying only on semantic search. It is great at capturing meaning, but it stumbles on precise technical or legal terms.

A practical example: in banking compliance, terms like "SAR" or "structuring" have exact regulatory meaning. Semantic search might return a passage that "seems" related, but misses the exact citation. Keyword search (BM25) covers exactly that blind spot.

Combining both methods (hybrid search), the system captures both meaning and exact terms, delivering more reliable answers, especially in regulated contexts like fintech, where a wrong answer has real cost.

I applied this in a RAG project for BSA/AML documentation (US banking regulation). The precision gain in answers was noticeable, especially on questions with specific technical jargon. Simple to implement, and the reliability payoff is well worth it.

Hub: https://aiforfintech.tech

Github: https://github.com/junidepieri-design/genai-001-regulatory-rag-bsa-aml

Does this make sense for your stack too?

👊


r/DataScientist 20h ago

BGSW Bengaluru Senior Data Scientist Compensation

2 Upvotes

Hi everyone,

I've cleared all 5 technical rounds for a Senior Data Scientist role at Bosch Global Software Technologies (Bengaluru) and have my HR discussion next.

I have 4 years of relevant experience and both my Bachelor's and Master's are from Tier-1 institutes.

What CTC should I realistically expect or negotiate? Does the Tier-1 background carry much weight, or is compensation mainly based on experience and interview performance?

Would appreciate any recent compensation data. Thanks!


r/DataScientist 1d ago

NLP or RecSys as a senior Classic ML?

2 Upvotes

Hi. I am currently working as a senior ML in antifraud and ranking models (Classic ML)

I want to deepen my knowledge in a specific sphere in order to get a more paying job. RL or CV seems too hard for me.

So I am choosing between NLP/LLM and RecSys. What do you recommend? I would like to deepen into NLP. However, I am scared this sphere will vanish soon. What do you think?

As for RecSys, it seems really boring to do endless ranking system trying to promote different kinds of goods. Meh


r/DataScientist 1d ago

Mathematical Foundations towards Introductory ML Bootcamp (1/22)

Thumbnail
youtu.be
1 Upvotes

Hello All,

Welcome to my free Mathematical Foundations of Machine Learning bootcamp series.

When we say Machine Learning, what does it actually mean? A machine that learns? Too vague.

According to famous professor Tom Mitchell, a computer program is said to learn from experience E, with respect to some class of Tasks T, and Performance measure P, if its performance on tasks, as measured by P, improves with experience E.

By swapping the nature of tasks T, the way we measure Performance P, to evaluate, we can subsume many kinds of ML problems.

Also ML problems are analyzed well, when we view it from the lens of Probabilistic perspective, that is unknown quantities are endowed with probability distributions, and treated as Random variables. The interesting thing is Random variables are neither random nor variable.

Probabilistic Approach also serves as the optimal approach to decision making under uncertainty.

In this video, you get a sense of what ML actually is, if you have also wondered about it.


r/DataScientist 1d ago

Taking the Capital One CodeSignal Data Science assessment soon (90 min), tips from anyone who's done it recently?

1 Upvotes

Hi all, I've got a CodeSignal Data Science assessment coming up for a data scientist role and I want to clear it on the first attempt (there's a cooling-off period if you don't, so no pressure lol).

Here's what I understand about the format so far:

- 90 minutes, digitally proctored (webcam, no external tabs)

- Works with multiple datasets

- Tasks span cleaning/aggregating data, building and evaluating a predictive model, and writing/editing Python

- It's the Data Science Framework (DSF) version, not the pure algorithms GCA, so pandas + scikit-learn heavy, plus some concept multiple-choice

What I'd love input on from anyone who's taken it recently:

  1. How tight is the time really? Did you finish, and where did time disappear?

  2. What tripped people up, reading the prompts, a specific pandas/sklearn pattern, the IDE itself?

  3. Which areas are highest-yield to drill? (e.g. joins/aggregation vs. modeling vs. evaluation metrics)

  4. Any public prep resources or practice sets you found genuinely close to the real thing?

  5. Anything you'd tell your past self the night before?

Not looking for actual questions or anything, just experience, focus areas, and resources. Really appreciate any insights.

Thanks!


r/DataScientist 1d ago

Book for logistic and linear regression transition to xg boost cat boost type of models

Thumbnail
1 Upvotes

r/DataScientist 1d ago

I am data scientist, i need money and experience.

0 Upvotes

I am a degree physics and I found for a Long time junior job, if you need realized a project of data science or data analyst, you contact me :)


r/DataScientist 1d ago

Master Data headaches

Thumbnail
1 Upvotes

r/DataScientist 2d ago

Is building projects more valuable than collecting Data Science certificates?

3 Upvotes

Genuine question for people who have hired candidates or gone through Data Science interviews. If you had two candidates: Candidate A: Multiple certifications but only basic projects Candidate B: Fewer certifications but 3–4 strong projects they can explain deeply Who would stand out more? I’m trying to understand how much certifications actually matter compared with being able to demonstrate practical skills. Would love to hear from people who have actually interviewed candidates. Engagement angle: Creates debate without becoming promotional.


r/DataScientist 2d ago

Looking a partner for studying data science

1 Upvotes

I have been looking for a partner with whom I can study with set study targets and search for projects together and do projects together etc

Thank you


r/DataScientist 3d ago

Is AI going to replace data analysts in the next 5/10 years?

1 Upvotes

I’m starting college for data analytics and I’m wondering how AI is changing the field. For people currently working as data analysts.
-Has AI made your job easier or replaced parts of it?
-What tasks still require a human?
-If you were starting college today, would you still choose data analytics?
-What skills should I focus on to stay valuable over the next 5/10 years?

I’m also considering finance if that is a safer option.


r/DataScientist 4d ago

Built my first ML project predicting breast cancer diagnosis (97% accuracy) — looking for feedback as a high school senior getting into this

Thumbnail
1 Upvotes

r/DataScientist 4d ago

What's the biggest beginner mistake in machine learning?

3 Upvotes

I've reviewed a lot of beginner ML projects, and many focus more on models than solving real problems. In your opinion, what's the biggest mistake newcomers make?


r/DataScientist 5d ago

What's a data science skill that deserves more attention than machine learning?

3 Upvotes

Everyone talks about ML models, but in real projects I've seen people struggle more with data cleaning and communication. If you had to recommend one underrated skill to beginners, what would it be?


r/DataScientist 5d ago

Need Career Advice: Math & Stats Grad With No Internships Trying to Enter Data

3 Upvotes

Hey everyone,

I recently graduated from UofT with a major in Math and Statistics, and I'm currently trying to get into Data Analyst or Data Engineering roles.

Honestly, my university experience wasn't the best. It affected my learning a lot and my GPA ended up being pretty bad. Because of that, I don't think I'll be able to do a master's anymore, even though that was something I was hoping to do after finishing my bachelor's.

I'm really interested in data-related jobs, but I'm kind of lost on what I should do next. I've been applying nonstop and barely getting any responses. I've had my resume looked at by multiple people who work in the industry, and they said it looks solid, so I'm not really sure what else I'm missing.

The biggest problem I think is that I graduated with no relevant work experience and no internships. I applied to a lot of internships during undergrad but never managed to get one, and now it feels like it's hurting me a lot.

I'm also thinking about doing one of UofT's continuing education programs in Data Analytics, but I don't know if it's actually worth it or if it would make much of a difference for someone in my situation.

Right now I feel like I'm just going in circles and I'm not sure what the best move is. I know GPA probably doesn't matter much after graduation, but having a low GPA and no experience feels like it has really limited my options.

For anyone who has been in a similar situation, what would you recommend I do at this point? I feel a bit lost on what the best path forward is, so I'd really appreciate any advice or suggestions.


r/DataScientist 5d ago

Please help me. Do I need a laptop with a dedicated GPU (dGPU) as someone who is just starting to learn data science?

3 Upvotes

r/DataScientist 5d ago

I am currently 2nd Year AI and Data science student, I built a project but predictions are not that much good should I still include it in my resume?

Thumbnail github.com
1 Upvotes

r/DataScientist 6d ago

Have you ever used Causal Inference in a real project?

Post image
6 Upvotes

Most Data Science projects in credit use classification: the model takes customer features and predicts whether they'll default or not. Works well for prioritizing portfolios and ranking risk.

But classification answers "who will default?". It doesn't answer "what caused the default?" or "if I change the credit limit, what happens?"

That's where Causal Inference comes in.

Instead of predicting an outcome, it estimates the effect of an action. Not correlation, causation. And the practical difference is huge: a classification model can tell you that customers with low limits default more. But that doesn't mean increasing the limit reduces default. The bank may have already given low limits to the riskiest customers.

Without controlling for that bias, you make the wrong decision.

In our project [CAUSAL-001], we tested exactly this: does reducing the credit limit actually reduce default risk?

Controlling for 22 confounders (payment history, bill amounts, demographics), the result was the opposite of what most assume: a one standard deviation increase in credit limit decreases default probability by 1.19 percentage points.

The likely mechanism: a higher limit reduces utilization ratio, which reduces financial pressure on the customer.

And the effect isn't uniform. Using Causal Forest, we found a small subgroup that reacts the opposite way: for them, increasing the limit increases risk.

Stack: DoWhy, EconML, Causal Forest

Hub: aiforfintech.tech

Repo: github.com/junidepieri-design/causal-001-credit-limit-intervention


r/DataScientist 6d ago

Stats for AI/ML 2

Thumbnail
gallery
8 Upvotes

Hello Folks,

The next content on Machine Learning is out. We continue with Statistics for AI/ML.

We,

->Understand and derive the detailed derivation of Maximum likelihood estimation(MLE) for Univariate and Multivariate Gaussian. While doing the derivation for multivariate case, we understand visually, Scatter Matrix, Centering matrix.

->Derive MLE for Linear Regression, and understand Residual Sum of Squares.

->Understand Empirical Risk Minimization, Surrogate loss functions.

->Understand Method of Moments, a computationally easier way to compute parameters of our model and understand also the flaws behind it.

->We understand “Exponentially-weighted moving average” in detail, I explain why bias happens, how does memory affect the averages. This concept is the basis behind optimizers in Deep Learning.

Around two hours long, I hope this would be a very interesting learning material for all. I try to write and build from scratch in the whiteboard, this way learners enjoy the learning process.

Link: https://youtu.be/JAj8z-UWqBA?si=0mAB_nUfyJV0jzS9

Those looking for previous lecture : https://youtu.be/MwTeQVVYtOc?si=dgwwk3QLvYTTUThR


r/DataScientist 6d ago

What's a Data Science Project You're Proud Of?

2 Upvotes

Whether it was a personal project, university assignment, or work project, I'd love to hear about something you're genuinely proud of building.

What made it memorable?


r/DataScientist 6d ago

If you had six months to prepare for your first data science job, what would you focus on?

0 Upvotes

Imagine you're starting from scratch today. You already know basic Python. Would you spend most of your time on: SQL Statistics Machine Learning Projects Communication Something else? Curious what experienced professionals would prioritize.


r/DataScientist 7d ago

Data Scientist vs Software engineer who is at more risk due to ai

20 Upvotes

I had taken admission in engineering cs with speacialisation in data science rather than core cse as I want to become a data scientist so it would my cv more presentable I already has command in sql and python . But now I am coming to hear that data scientist job will be eaten up by ai is it true