By Wright studied genetics at Harvard. While he was working at the University of Chicago, he interested in the inheritance of coat colour in guinea pigs. He found that it is nearly impossible to breed an all-white or all-coloured guinea pig, even the most inbred families which contradict the prediction of that time in which a …
How Statistics Became a Model-blind Data-reduction Enterprise? Karl Pearson
In the last blog post, we have covered Francies Galton and his Galton Board. In this post, we will talk about Karl Peason. Pearson was affected by Galton’s idea on correlation. He believes correlation is bigger than causation. Causation was reduced to nothing more than a special case of correlation. He said, “That a certain …
How Statistics Became a Model-blind Data-reduction Enterprise? Francies Galton.
Date: 5 June 2021 This is Chapter 2 of the book, The Book of Why. This chapter is an account of the history of statistics and how it departs from understanding causation to only correlation. I personally did not validate the accuracy and completeness of the stories in the Chapter. I feel that the Author …
If causation is not correlation, then what is it?
Date: 30 May 2021 I am currently reading a book called The Book of Why. I just finished Chapter 1, The Ladder of Causation, and would like to give you a quick summary on what I have learnt. In the recent advancement and success of machine learning and artificial intelligence, it seems that many problems …
Some Beginner Resources for Learning Data Visualization in Tableau
Date: 06 Jan 2021 Recently, I am preparing the launch of Tableau in my company. I have, therefore, collected some useful resources and want to share with my readers. General Learning Path Understand the terminology used in Tableau You may refer to this link. Understand the Tableau environment, meaning the functions of buttons and cards …
Data Governance V.S. Data Engineering V.S. Data Analysis
Date: 31 Dec 2020 Like most of you, when I tried to explore the career in data field, I want to be a data analyst. It is the sexiest job in 21st Century (according to Harvard Business Review. [an article titled: Data Scientist: The Sexiest Job of the 21st Century]). At that time, I didn’t …
Predicting House Price in Hong Kong #6
It has been a while since I made the last git commit in the project repository. Although I am a bit busy with my daytime job recently, I will keep the side project rolling. This blog post is about my recent reflection on the project direction. There are two goals I want to achieve in …
Predicting House Price in Hong Kong #5
in #4, I talked about how to find the hidden APIs of Centaline Android App. In this article, I am going to show you how to write a scraper to scrape the data. You can find the Scrapy Crawler in this Git directory. 3 Levels of Data The scraping was done in 3 levels. First, …
Predicting House Price in Hong Kong #4
Date: 26 July 2020 In #3, I faced difficulty in having a lot of missing values in the property transaction data. After searching the web, I found that Centaline claimed they have spent 10 million HKD to fill the missing values. Maybe, I should try scraping data from Centaline. Luckily, I have scrapped Centaline data …
How to Extract Data from Text Columns?
Imagine you need to extract the price from the “sell_price” column, what would you do? Before reading the solutions, leave a comment below! ad_cat sell_price 住宅 屋苑 村屋 售 358 萬 住宅 屋苑 售 960 萬 住宅 屋苑 售 465 萬 住宅 屋苑 售 760 萬 住宅 屋苑 售 650 萬 住宅 屋苑 單幢式大廈 售 …
