Saturday, April 18, 2009
I saw my thesis Supervisor yesterday and have not slept since.
Friday, April 10, 2009
I joined the International Association for Statistical Computing.
It is the mission of the IASC to link traditional statistical methodology, modern computer technology, and the knowledge of domain experts in order to convert data into information and knowledge.
Statistical Computing is meant here in its broad sense as Statistics in the Communication and Computer Age and it contains a rich variety of research topics in almost every branch of statistical inquiry.
The topics focused on by the IASC include computational statistics, statistical software, exploratory data analysis, data mining, pattern recognition, statistical graphics and data visualisation, statistical data bases, and related fields.
The IASC works to share and advance methods and techniques related to these topics by facilitating the exchange of information and by stimulating evaluations, research, and development in all areas of statistical computing. It is the aim of the IASC to foster world-wide interest in effective statistical computing and to exchange technical knowledge between researchers world-wide.
IASC promotes collaborative efforts within international, national, regional and other organizations and institutions having similar aims.
I am going to ProQuest, IEEE and ACM databases again for ten more items for background literature..
Thursday, April 09, 2009
I can read in the country data sets in two lines of code.
I figured out how to read in the country data sets in two lines of code. The problem is I need 228 of these pairs of lines. To do this I turn to the automated features of "fill down" in Excel. I am learning that in SAS one can do this with macros. In Excel I have created the first line of code where I set a specific counter based on the position of the countries data in the large file. Once this so called counter or index of numbers is set the second line is a as.data.frame line that reads the large file but only reads the rows( as indicated by the counter/index set of numbers) needed for that country. So today I am going to use Excel to write the second line of code 228 times.
I am still using Inference for R and was mistaken in a sexist way about the representative changing her last name. Of course, many women have two last names and that was all it was. I have only completed the first tutorial for Inference for R and have not yet tried the studio application. This promises to allow breaks while running code.
At this point in testing this Inference for R software, after I have the second lines of code completed I will run this and try printing the Inference for R word document and adding a write up. I may use some of this blog for the write up. I will do this in word. I will after that try to make a Power Point for the conference and start to write up this part of the research. Ideally I will have only a few slides devoted to this part of the project but that show some of the code.
Then it is off to Google or back to the US census site to find data about major industries by country.
Saturday, April 04, 2009
Beautiful quote from the Rae years by Lilith Finkler from Candian Woman Studies
The politics of appropriation are such that a privileged few manage to determine subjects of study and gain recognition as "experts" on a community outside their own. This process becomes particularly insidious when so - called "progressive" individuals claim to be allies of an oppressed group and in so doing, constantly speak on their behalf. This disempowers the people directly affected by a particular experience, since they have less "credibility" and less access to privilege.Finkler, Lilith. Canadian Woman Studies. Downsview: Summer 1993. Vol. 13, Iss. 4; pg. 72.
Inference for R software. Quality issues explored
Practically twenty books or hundreds of journals articles will not work.
My searching of ProQuest that started a year ago is now filtered down to 79 references.
Friday, April 03, 2009
I am co-presenting a paper at a work conference.
Socio-economic Conference 2009
Baby-boom research
Thursday, April 02, 2009
I have made it to ten references now.
Monday, March 30, 2009
I installed Microsoft Project 2007 and will use it to plan my thesis work.
Sunday, March 29, 2009
I plan to find five more classic literature items today
Saturday, March 28, 2009
Progress with spring paper: I can read by 21 rows now and have all the population data into 18 datasets
I have now read all my data into 18 age band data sets and just need to now join these 18 data sets and then split these every 21*23 rows into 228 data sets and then again split this into 23 yearly tables. So the end result should be 227 countries and one world data sets with population for both genders and population for each of male and female by 17 age bands and total all ages, and 23 yearly tables for each country.
This is giving me an idea for the analysis. This idea is to use the 18 data sets and call these the age band data sets(18). Then I want to have 227 country and one world data sets and call these the country data sets(228). Then also have 23 data sets for years and call these the years data sets(23). Then I will work comparing these data sets for my populations analysis.
In fact, rather than join these data sets I have now, the age band data sets to get the other data sets, I should instead simply read in the data for each data set. So my next reading in will be for the 227 countries and one world data sets. Then the third step will be to read in the data into years data sets.I also need to start looking at a data source for finding the top three industries in each of the 228 countries. May be one google search like I did to find the population data will work.
I have started using Inference for R software and can code with this documentation software now.
I was offered a chance to test out Inference for R. This software it seems interfaces R with Microsoft Office. I have so far used it in both Word and Excel. Because it is an interface it is faster than copying and pasting code into the R command line window and also is faster because when you change code and run that changed code as in debugging you do not lose the code typed on the command line. I had in the past changed my code on the command line and would have to copy my successful code back to my text editor to keep a record of what I was doing.
One big problem for Inference for R is that it is only available on one platform. I am having to move back to the Windows world of computers because of statistics software like Inference for R, SAS and Excel and also because of the game Eve Online which is mostly a windows based game.
My computers are working well. I am having to use Windows at school and work.
My eMac is functioning fine and I even found the DVD drive working again. The Macbook although it is filthy is working fine as my usual daily machine now in the living room, The new home built PC with Windows XP 64 bit and Debian Linux is also working fine now. The new Dell Mini 9 Netbook is fine and I carry it around all the time but it is not getting much use but it is very useful to have it if needed. I used it at a community group last week to keep minutes of a meeting.
I am thinking of buying a Windows laptop that I would build from an OCZ laptop skeleton. This PC laptop would also be running Windows XP 64 bit and Debian 4.2. I am also thinking of buying an iMac 20 inch desktop. I am looking at used MacBooks and Powerbooks and iBooks for a friend and perhaps one for myself for music recording. I am looking in the 200-400 dollar range for these. These would be running Tiger not Leopard for an OS.
Friday, March 20, 2009
Getting books and papers organized: Shelves I use for books.
Literature search and studies update
After meeting my supervisor I felt confident that my thesis is progressing well. I am focused on the literature search and my paper for a conference in social economy in two months. These are the two basic tasks at present. I will need to find the twenty classic references for my topic. Then I will spend the summer term reading these references. I have four items in mind so far for this twenty item list.
In health reading I am reading about stress management. In entertainment reading I am reading two books about guitar. In programming I am studying SAS, Excel and have numerous computer "language" books. I will need to learn to program in Java or C++ this summer.
Thursday, March 19, 2009
I was unable to import the data in R or in SAS so I am working in Excel.
Sunday, March 15, 2009
Progress with my paper for the May graduate conference in social economy.
I have created columns next to the total population and male and female population counts columns that subtract a given age band from the previous age band. Thus if these new columns are negative it means a given age band is larger than the younger age band before it. For instance, if there are 300,000 persons aged 5 to 9 years old and there are only 200,000 babies aged 0-4 years old in a country in say 1996 then the next columns shows - 100,000. I then used conditional formatting to highlight the new column if a value was negative. This has given me a way of visually finding any suspected baby booms. But I still desire R code that will scan these new columns and produce a data set of only the population tables where these negative values occur. At this point, as well I have been offered the use of a documentation software to document in Office 2007 format, my use of R code.
I have some idea of the required indexes in the data set to use in R loops or array indexes. These are the re-occurrence rows of data in the large data set. In other words starting with the World population data tables then going from country names starting at A then down to Z we have repeated tables starting in 1996 to 2010 for each year and then 2015 and basically every five years projected until 2050. So there are tables in two dimensions, country names and then years. I will limit my searches to 1996 to 2005 at first but then if I am choosing a country as having booms in this decade of years 1996 to 2005, I will come back to the projections for the future.
I also need to get the path's right to open data files using R on windows. So once this path problem is solved I can move ahead with trying index's to parse the large data set into a smaller data set with only population tables that indicate a suspected baby boom.
