This paper shares my takeaways, best practices and mitigation steps to mitigate unfairness and bias while designing machine learning algorithms.
Data science projects go wrong either due to flawed models or insufficiently/ incorrectly trained algorithms or emergent bias on new/ unanticipated contexts. Fairness is a human, not a mathematical decision, grounded in shared ethical beliefs. While machine learning does not make decisions based on feelings and emotions, it does inherit a lot of human biases leading to disparate impact. In this era where consequential decisions are algorithm-based it is imperative that they are fair, not perpetuated without users knowledge. …
This paper utilizes Fargo Health Group dataset to forecast the demand for heart examinations expected in 2014 for Abbeville Health Center. It outlines how a business problem can be solved using a data-driven decision making approach and explains the methodology, model leveraged, ethical implications and recommendations for Fargo Health Group.
Fargo Health Group faces the following business problems:
Diamond pricing involves a complex mechanism influenced by multiple factors such as carat, cut, color and price. This article analyzes the correlation between these factors and depicts with visualizations.
Exploratory data analysis
R diamond.csv dataset includes approximately 54K observations with 10 variables including carat, cut, color, clarity, depth, table, price, x (length in mm), y (width in mm) and z (depth in mm). Overall a clean dataset with no missing values or messy data.
Structure of the dataset (R lang)
People v. Collins was a 1968 American robbery trial noted for its misuse of probability and as an example of the prosecutor’s fallacy. (Wikepedia, 2021)
After a mathematics instructor testified about the multiplication rule for probability, though ignoring conditional probability, the prosecutor invited the jury to consider the probability that the accused (who fit a witness’s description of a black male with a beard and mustache and a Caucasian female with a blond ponytail, fleeing in a yellow car) were not the robbers, suggesting that they estimated the odds as:
Black man with beard 1 in 10 Man with mustache1…
Major customers served by Transport for London include domestic and international passengers on all the modes of transportation for its overground, underground services from cycles to river services and everything in between. In addition, TfL customers include contract operators of a bus, river services, and tram.
With TfL’s slogan of “Every journey matters,” they have leveraged Big Data analytics to improve customer engagement. Few ideas include:
Two major customer groups of Airbnb include travelers (guests) looking for accommodation and patrons (hosts) looking to offer temporary property rentals.
Ideas to improve customer experience:
In today’s era, any company can become a Big Data company. This article explores similarities between John Deere and Apple Big Data strategies. Few areas are:
John Deere’s strategy with ag-analytics focuses on manufacturing farm equipment that is essentially software with engines. Farm equipment is now more loaded than a standard car. John Deere’s vision is to leverage Big Data analytics and AI so one day farmers will be able to use a fleet of autonomous robotic tools from a command room assisted with a smaller workforce. …
Microsoft’s primary revenue strategy focuses on leveraging Big Data to gain market share with its enterprise-ready Big Data Solution — HDInsight, competing with Google Cloud and Amazon AWS. Azure HDInsight based on open-source Hadoop, Spark, Kafka framework offers ETL, data warehousing, IoT, and Machine Learning capabilities. With PostgreSQL, it offers Oracle database compatibility tapping into clients relying on on-premises Oracle database (Azure Storage).
When we talk about search engines, Google pops as a default. However, even before the buzzword of Big Data came into being, Microsoft adopted Big Data with its proprietary web search engine — Bing, rebranded from its…
This article explores how Apixio leverages Big Data, AI, machine learning and deep learning algorithms to analyze structured and unstructured sources of patient records. It’s mission is to use Artificial Intelligence (AI) to change the way healthcare is measured, care is delivered, and discoveries are made.
Apixio’s value proposition is to learn how to make sense of clinical information at scale to provide better individual care and gain insights on overall population health by mining Big Data. It pitched the same…
This article covers how Experian leverages Big Data and analyzes one of Experian’s products.
Exec Director StratEx - I bring to the table blend of data science, finance and strategy management skills with 20+ years of experience in insurance & fintech.