Lihan (Delta) Zhang

Week 2 of CUNY Tech Prep has been enlightening! I delved deep into data cleaning and discovered the power of using agg() to extract key statistical summaries from a data frame - mean, max, min, median, sum, and more. Moreover, I was introduced to Tableau, a dynamic data visualization tool. I even watched a couple of how-to videos to enhance my understanding. Excitingly, I am now exploring Machine Learning. If you have any valuable resources to recommend on this topic, I would greatly appreciate it.

7

2 Comments

I think this looks like an excellent resource. My position requires me to be ambidextrous with R and Python, and I am stronger in R for many things. The book is designed as a course "to be read from beginning to end", and while the material is not complex, a course like this is sure to fill in gaps.

6

#causalinference is a critical part of a data scientists work. In an effort to help a VP and faculty member discern whether universalizing an intervention, for free and therefore at a cost to the college, was beneficial, I recently applied propensity score matching (PSM) to analyze the effects of the intervention. First, here's why I chose it and why it's important for a data scientist to learn. The gold standard in the world of research is Randomized Controlled Trials (#RCTs); however, it is not always feasible to conduct RCTs in the real world. Enter -->🚪 PSM. PSM uses logistic regression to calculate the propensity score (PS), (most commonly, though the #MatchIt package in #R allows for other options like RF, NN, CART, GBM, to name a few). The PS is the probability a subject (or customer) receives the treatment given their covariates and is calculated to identify a control group that is statistically similar to the treatment group. Since this is a real-world experiment, the treatment is the "known" cohort, while the control group is unknown. The general consensus to properly carry out PSM is to include as many covariates as possible because you are trying to limit confounding variables from introducing bias into the estimates. In fact, this is one of the great criticisms of PSM that the original authors Rosenbaum and Rubin cautioned practitioners against in their seminal paper over forty years ago (1983). When properly carried out, PSM matches participants or customers with similar propensity scores in an effort to reduce selection bias and enable a valid comparison between treatment and control groups. By balancing covariates between groups, the comparison acts more like an RCT, which provides a clearer picture of the intervention's effectiveness. After the two groups have been identified, additional analysis can be carried out to measure the effects of the treatment or intervention. #datascience #machinelearning #propensityscorematching

1

1 Comment

Being an amazing data scientist != you can lead the team! You can't just: - use more tools - put in more hours - learn more models - fulfill more requests - juggle more projects - respond faster to email - complete more "story points" (whatever the hell that means) You're already doing this, and your boss is telling you "you're not ready." They won't tell you why. They won't tell you what you're missing. And even if they did, there's no guarantee they'll have a role or budget to promote you. 99% of the time, they don't even know what you're missing, anyway. So you think you'll have better luck elsewhere. But no one will give you your first shot. You're stuck. And it's maddening! Ever seen a job that says "looking to hire an IC into their first manager role?" Me neither. Being an amazing data scientist does not mean you can lead the team! Why? Because... A DS leader's job is NOT to DO data science! Read that last line again. "Doing" data science is the data scientist's job. The DS leader's job is to ADVANCE the FUNCTION of data science. Advance towards what? - AI-driven cost savings (and) - AI-driven revenue growth You do this by 1. building partnerships across the business (in order to) 2. embed AI throughout the operating model (and) 3. launch AI-driven products That's it. That's your job. Print this out and put it on your wall if you want. This is the correct order of operations for all data science functions that aren't AI-first startups. Sometimes this means writing more code. Sometimes this means reading more papers. Sometimes this means learning more models. Often this means more MLOps. Often this means more meetings (for you). Often this means more focus on filling the team's skill gaps. If "doing" data science is the best way RIGHT NOW to advance the function, then you MUST do it. But often you're just doing it because it's in your comfort zone. To lead is to embrace discomfort. To lead is to Advance the Function. Never forget! BTW, if you liked this, a repost would mean the world to me. #thinksenior

76

5 Comments