Data Science_W2
Identify all questions that you attempted in this template
1. For each of parts (a) through (d), indicate whether we would generally expect the performance of a flexible statistical learning method to be better or worse than an inflexible method. Justify your answer.
(a) The sample size n is extremely large, and the number of predictors p is small.
(b) The number of predictors p is extremely large, and the number of observations n is small.
(c) The relationship between the predictors and response is highly non-linear.
(d) The variance of the error terms, i.e. σ2 = Var(), is extremely high
5. What are the advantages and disadvantages of a very flexible (versus a less flexible) approach for regression or classification? Under what circumstances might a more flexible approach be preferred to a less flexible approach? When might a less flexible approach be preferred?
6. Describe the differences between a parametric and a non-parametric statistical learning approach. What are the advantages of a parametric approach to regression or classification (as opposed to a nonparametric approach)? What are its disadvantages?
Q2 Textbook Applied Questions – Attempt with Python
8. Exploratory Data Analysis: College data set: College.csv. It contains a number of variables for 777 different universities and colleges in the US. Do all the exercises in Python:
8a. Read the csv file with pandas
8b.Fix the first row as row headers
8c.
- produce a numerical summary of the variables in the data set.
- produce a scatterplot matrix of the first ten columns or variables of the data.
- produce side-by-side boxplots of Outstate versus Private
- Create a new qualitative variable, called Elite, by binning the Top10perc variable and divide universities into two groups based on whether or not the proportion of students coming from the top 10 % of their high school classes exceeds 50 %
- Produce some histograms with differing numbers of bins for a few of the quantitative variables: Room.Board’,’Books’, ‘Personal’, ‘Expend’
- Examine the elite schools more closely.
Q3 Textbook Applied Questions – Attempt with Python
9. Exploration with Auto.csv data.
Make sure that the missing values have been removed from the data.
(a) Which of the predictors are quantitative, and which are qualitative?
(b) What is the range of each quantitative predictor?
(c) What is the mean and standard deviation of each quantitative predictor?
(d) Now remove the 10th through 85th observations. What is the range, mean, and standard deviation of each predictor in the subset of the data that remains?
(e) Using the full data set, investigate the predictors graphically, using scatterplots or other tools of your choice. Create some plots highlighting the relationships among the predictors. Comment on your findings.
(f) Suppose that we wish to predict gas mileage (mpg) on the basis of the other variables. Do your plots suggest that any of the other variables might be useful in predicting mpg? Justify your answer.
Q4 Textbook Applied Questions – Attempt with Python
10. Exploration with Boston.csv data
a) How many rows and columns in the data set? What do the rows and columns represent?
(b) Make pairwise scatterplots of the predictors (columns) in this data set. Describe findings.
(c) Are any of the predictors associated with per capita crime rate? If so, explain relationship. (d) Do any of the suburbs of Boston appear to have particularly high crime rates? Tax rates? Pupil-teacher ratios? Comment on the range of each predictor.
(e) How many of the suburbs in this data set bound the Charles river?
(f) What is the median pupil-teacher ratio among the towns in this data set?
(g) Which suburb of Boston has lowest median value of owner occupied homes?
What are the values of the other predictors for that suburb, and how do those values compare to the overall ranges for those predictors? Comment on your findings.
(h) In this data set, how many of the suburbs average more than seven rooms per dwelling? More than eight rooms per dwelling? Comment on the suburbs that average more than eight rooms per dwelling.
Hint – several github sites have the complete solution in python e.g.
