Fun with Complex Quadratic Polynomial Julia Set by Using Julia Programing

August 5, 2022 at 4:39 PMSteven Wang

Posted in:


Inter Rater Reliability Study with Cohen’s Kappa and Fleiss’ Kappa

April 23, 2018 at 2:43 PMSteven Wang

Posted in: NLP


Magic Quadrant for Business Intelligence and Analytics Platforms 3 Year in A Row

February 5, 2016 at 9:11 PMSteven Wang

Comparing to past 2 years, the latest published Gartner Magic Quadrant for Business Intelligence and Analytics Platforms has some very interesting changes.

There are only 3 big players stay in the first group: Tableau, Qlik and Microsoft. Given that the MS did a fantastic job on the new Power BI platform, I'm not surprising MS is waken up and rising in this space. I was surprised that Microstrategy, IBM, SAP and Information Builders are fallen out the first group. Particularly, Oracle is dropped off the radar.

This is getting more and more interesting and look forward to how all these players doing this year.

Magic Quadrant for Business Intelligence and Analytics Platforms 3 Year in A Row



Posted in:


Using Shiny Server to Deploy Interactive Visualization Part 1: Data Exploratory Analysis of Red Wine Data

February 3, 2016 at 4:11 AMSteven Wang

Posted in:


Tackling the Windows Installation Issue of R packages: slidify, slidifyLibraries, and rCharts

January 8, 2016 at 11:55 PMSteven Wang

Slidify is an R package helps to create and publish beautiful HTML5 presentations from RMarkdown. Slidify package is fantastic for creating reproducible presentations.

slidifyLibraries contains external resources used by Slidify.

rCharts is an R package to create, customize and publish interactive javascript visualizations from R using a familiar lattice style plotting interface.

All of these three packages are created and maintained by Ramnath Vaidyanathan of McGill University.  Since all these three packages are not on CRAN and needs to be installed from github using the Hadley's devtools package.





As Ramnath mentioned in one of this Slidify workshop, While the installation process from github is relatively painless for Mac/Linux/Ubuntu users, for the windows users it is not that luck. I have to confess that when I used the above code to install these packages it caused a chaos for me. Not only that the package was not installed properly but also some other installed packages were broken!!!

Thiago Silva’s post Installing Slidify on a Windows Machine has provided his advice on how he successfully made a workaround. I will give a step by step guide on top of his blog, which I have successfully installed these 3 packages in window 7 and windows 10 machines.

1.       Download packages zip files

a.       Download slidify zip file from Github 

b.       Download slidifyLibraries zip file from Github 

c.       Download rCharts zip file from Github 

2.       Put zip files into a folder, say, c:/packages

3.       Unzip packages and you will notice that all packages have a “-master” suffix in the folder name. “-master’ will need to be removed as below:

4.       Now we will use the build utility from dvtools package to convert a package source directory into a single bundled file. If set binary = FALSE, this creates a tar.gz package that can be installed on any platform, provided they have a full development environment. Now we will use the below code to build packages:


build("C:/packages/slidify/", binary=FALSE) # Change C:/packages/ to your folder name

build("C:/packages/slidifyLibraries/", binary=FALSE) # Change C:/packages/ to your folder name

build("C:/packages/rCharts/", binary=FALSE) # Change C:/packages/ to your folder name

5.       Once packages are built successfully, we use the below code or alternatively we can use the RStudio to install these packages:

a.       R Script code:

# Change C:/packages/ to your folder name

install.packages("C:/packages/slidify_0.5.tar.gz", repos = NULL, type="source")

# Change C:/packages/ to your folder name

install.packages("C:/packages/slidifyLibraries_0.3.1.tar.gz", repos = NULL, type="source")

# Change C:/packages/ to your folder name

install.packages("C:/packages/rCharts_0.4.5.tar.gz", repos = NULL, type="source")

b.       Install from RStudio, go to manu: Tools, Install Packages… as below:

This has successfully installed these three packages: sidify, slidifyLibraries, and rCharts in windows environment.

I hope this helps and thanks for reading.

Posted in: R

Tags: , ,

A Simple Way of Quantiling Process in T-SQL by using Window Function and Ceiling Function

December 11, 2015 at 6:35 AMSteven Wang

In business analysis, it is very common to break sales into different quantile groups and then it can be used for segmentation purpose. The most commonly used quantiles are:

  • Quintiles which creates 5 equal size groups
  • Deciles which creates 10 equal size groups

Occasionally, Ventiles (20 groups) and Percentiles (100 groups) will used.

Quantiling process is very powerful when 2 dimensional of sales data is available, like product sales and market sales. In business it is very common to use quantile matrix to perform the account segmentation. An example of segmentation definition can be based on a 2 X 2 deciling matrix like below:

Even this is a very common process, it is indeed not a straightforward simple calculation to get quantile values in SQL server. There are many ways we can do to calculate such, but I found that use window function in conjunction with CEILING function is the easiest way to calculate quantile value. Let’s try. I will use a deciling process to start with.

The first step to calculate decile value for a specific object (Account, Product etc) is to calculate cumulative values and then calculate the cumulative percentage.

This will generate the cumulative product sales value and its cumulative percentage. the top 10 records as below:

Now the question is how can we assign a decile value based on the cumulative percentage value? there are many ways to do it. But as I found that there is a very simple way to do it by using SQL CEILING function:

You can randomly check the results:

Based on this deciling calculation, we can generalize the quantiling calculation on any quantile number between 1 and 100.

I hope this tip is useful and please see the attached scripts if you want to have a try.


Quantile_Calculation_Script.sql (1KB)

Posted in: T-SQL | Window Functions


Prediction of Correctness on Barbell Lifting Exercise Based on Accelerometer Data

November 24, 2015 at 2:45 AMSteven Wang

Posted in: Data Cleaning | Big Data | Data Load | Machine Learning | Random Forest | RPART | Predictive Analytics | R

Tags: , ,

Embedded Power View Report Rendering Issue in Different Browsers

March 18, 2015 at 2:19 AMSteven Wang

Power View Report embedded view is indeed a very nice feature and opens up a big opportunities for ISVs and companies and individuals to share data and visualizations. Please check this blog, Embed Power View Interactive Reports in Your Blogs and Websites, for more details about how to embed a power view report.

While this is great news, I also found that there are some frustrations. The embedded view is rendered nicely in IE 9 but there are issues with IE 11 and other main browsers. 

I have put together a few mains issues I found. If you have same issue and different issue please share your experience with me. Thanks.

Posted in: Data Explorer | Power BI | Power View


SSRS 2012: Report Server cannot load the TERADATA/SQLPDW Extension Error

May 8, 2013 at 3:49 PMSteven Wang


In the SQL Server 2012 reporting service server event log, you might have these 2 errors:

1.       Report Server cannot load the SQLPDW Extension

2.       Report Server cannot load the TERADATA Extension

While you are not using the TERADATA/SQLPDW functionalities in your report, basically you can ignore these errors. This is because by default TERADATA and SQLPDW data extension is turned on in the reporting server configuration file.

Even you are not using the TERADATA/SQLPDW functionalities, it is still good idea to suppress the error logged into the event log.

Open the reporting service configuration file rsreportserver.config under your SSRS installation folder. If you installed SSRS 2012 by default, the file will be under:

C:\Program Files\Microsoft SQL Server\MSRS11.MSSQLSERVER\Reporting Services\ReportServer\

If you locate the Data Section you will see the below yellow highlighted data extension TERADATA and SQLPDW is turned on.

If you comment out these 2 extensions as below and save, those error messages will no longer be generated.


Posted in:

Tags: ,

SQL Server 2012 MDS Windows Server Minimal Requisites

April 23, 2013 at 4:49 AMSteven Wang

In order to make SQL Server 2012 MDS functioning properly under Windows Server 2008 R2, the minimum windows features must be met.

1.       1. Add Application Server role

The following role servers for Application Role need to be included:

·         .NET Framework 3.5.1

·         Web Server (IIS) support

·         HTTP Activation

As illustrated in the diagram below (ticked services):


1.       2. Add Web Server (IIS) role

Make sure that the yellow highlighted services are ticked.

1.     3.  Check additional windows features

Make sure the Features listed below (yellow highlighted) are installed:

Posted in: Master Data Service