Analysis with Programming

Another powerful procedure of SAS, my favorite one, that I would like to share is the PROC IML (Interactive Matrix Language). This procedure treats all objects as a matrix, and is very useful for doing scientific computations involving vectors and matrices. To get started, we are going to demonstrate and discuss the following: Creating and Shaping Matrices; Matrix Query; Subscripts; Descriptive Statistics; Set Operations; Probability Functions and Subroutine; Linear Algebra; Reading and Creating Data; Above outline is based on the IML tip sheet (see Reference 1). So to begin on the first bullet, consider the following code:

Analysis with Programming has recently been syndicated to Planet Python . And as a first post being a contributing blog on the said site, I would like to share how to get started with data analysis on Python. Specifically, I would like to do the following: Importing the data Importing CSV file both locally and from the web; Data transformation; Descriptive statistics of the data; Hypothesis testing One-sample t test; Visualization; and Creating custom function. Importing the data This is the crucial step, we need to import the data in order to proceed with the succeeding analysis. And often times data are in CSV format, if not, at least can be converted to CSV format. In Python we can do this using the following codes:

So I started using SAS® University Edition which is a FREE version of SAS® software. Again it's FREE, and that's the main reason why I want to relearn the language. The software was announced on March 24, 2014 and the download went available on May of that year . And for that, I salute Dr. Jim Goodnight . At least we can learn SAS® without paying for the expensive price tag, especially for single user like me. The software requires a virtual machine, where it runs on top of that; and a 64-bit processor. To install, just follow the instruction in this video . Although the installation in the video is done in Windows, it also works on Mac. Below is the screenshot of my SAS® Studio running on Safari.

We are going to explore the basics of Statistics using Python. And we'll go through the following: Importing the data; Apply summary statistics; Other measures of variability (variance and coefficient of variation); Other measures of position (percentile and decile); Estimate the Skewness and Kurtosis; and bonus, Visualize the histogram; Data -- volume of palay (rice) production from five regions (Abra, Apayao, Benguet, Ifugao, and Kalinga) of the central Luzon, Philippines. To import this, execute the following: To check the first and last five entries of the data, use head() and tail() methods, respectively; and to apply the summary statistics, use the describe() method,

The first 40 raw data of Systolic Blood pressure (in mmHg) of Simplified Biostatistics by Abubakar S. Asaad is utilize in the demonstration below. Output: C Codes: C++ Codes:

Output:   C Codes: C++ Codes:

Output: C Codes: C++ Codes:

Output: C Codes: C++ Codes:

Interval estimation of the population mean can be computed from functions of the following R packages: stats - contains the t.test ; TeachingDemos - contains the z.test ; and, BSDA - contains the zsum.test and tsum.test . The t.test of the stats package is a student's t test, and is use when raw dataset is given. The same case for z.test , but this function is specifically for z-test of known population standard deviation. When dataset is not given and only the summary statistics (mean, and standard deviation) are presented, then the appropriate functions are zsum.test or tsum.test . Note that, t.test and tsum.test are functions of the same statistical test, and that of z.test and zsum.test . Consider the example below, Example 1 . The 2012-2013 SASE scores of the 33 random students from College of Science and Mathematics (CSM) of MSU-IIT were recorded: 84, 93, 101, 86, 82, 86, 88, 94, 89, 94, 93, 83, 95, 86, 94, 87, 91, 96, 89, 79, 99, 98, 81, 80, 88, 100, 90, 100,...

Skewness and kurtosis in R are available in the moments package (to install a package, click here ), and these are: Skewness - skewness ; and, Kurtosis - kurtosis . Example 1 . Mirra is interested on the elapse time (in minutes) she spends on riding a tricycle from home, at Simandagit, to school, MSU-TCTO, Sanga-Sanga for three weeks (excluding weekends). She obtain the following data: 19.09, 19.55, 17.89, 17.73, 25.15, 27.27, 25.24, 21.05, 21.65, 20.92, 22.61, 15.71, 22.04, 22.60, and 24.25. Compute and interpret the skewness and kurtosis. Interpretation : The skewness here is -0.01565162. This value implies that the distribution of the data is slightly skewed to the left or negatively skewed . It is skewed to the left because the computed value is negative, and is slightly, because the value is close to zero. For the kurtosis, we have 2.301051 implying that the distribution of the data is platykurtic , since the computed value is less than 3. Graphical illustration of the da...

The measure of relative variability is the coefficient of variation (CV). Unlike measures of absolute variability, the CV is unitless when it comes to comparisons between the dispersions of two distributions of different units of measurement. In R, CV is obtained using the cv function of the raster package. Example 1 . Below are the mean and standard deviation of the number of hours spent by Jacob every time he study the Stochastic Process with the corresponding scores he got out of the 100 items. Basing from this data, should one say that the number of hours he spent in studying is more variable than his exam scores, or the other way around?  Variable   Mean   Standard Deviation   Study Hours    25   2.6  Scores 69 5.3 To determine this, we use the function below And thus, Interpretation : It is very clear from the computed CV that, the study hours is more variable than the exam scores, even though the st...

Measures of absolute variability deals with the dispersion of the data points. This include the following: Range - range ; Interquartile Range - IQR ; Quartile Deviation; Average Deviation; and, Standard Deviation - sd . These measures of variability are restricted to uniform units of measurement when comparing two distributions. Example 1 . The heights (in centimetres) of the 17 BS Stat students in section A23 of Statistical Inference under Dr. Supe were recorded. The data are the following: 151, 160, 162, 155, 154, 154, 153, 168, 169, 153, 158, 166, 152, 157, 150, 169, and 167. Compute the range, interquartile range, quartile deviation, average deviation, and standard deviation. The range is computed using the function range , while the interquartile range is obtained by IQR . Thus,

The measures of position such as quartiles, deciles, and percentiles are available in the quantile function. This function has a usage, where: x - the data points; prob - the location to measure; na.rm - if FALSE , NA (Not Available) data points are not ignored; names - for attributes, FALSE means no attributes, hence speeds-up the computation; type - type of the quantile algorithms; and, ... - further arguments. Example 1 . The junior BS Stat students of MSU-IIT have the following SASE scores: 88, 84, 83, 80, 94, 90, 81, 79, 79, 81, 85, 87, 86, 89, and 92. Determine and interpret the quartiles of these scores. Interpretation : Therefore, $Q_1$=25% implies that, 25% of the SASE scores fall below or equal to 81.0, while the other 75% of it is above 81.0. $Q_2$=50% is the median, and thus half of the scores are below or equal to 85.0, while the other half, are above 85.0. $Q_3$=75%, implies that three-fourth of the data are below or equal to 88.5, while the r...

Mean in R is computed using the function mean . Consider the scores of 20 MSU-IIT students in Stat 101 exam with hundred items: 70, 78, 66, 65, 50, 53, 48, 88, 95, 80, 85, 84, 81, 63, 68, 73, 75, 84, 49, and 77. Compute and interpret the mean and median. Interpretation : Therefore, the average score of the students is 71.6, and half of their scores are less than or equal to 74, while the other half are greater than 74.