This is a story about what happened when I tried to use big data to help repair my local public schools. I failed. And the reasons why I failed have everything to do with why the American system of standardized testing will never succeed. A few years ago, I started having trouble helping my son with his first-grade homework. I’m a data-journalism professor at Temple University, and when my son asked me for help on a worksheet one day, I ran into an epistemological dilemma. My own general knowledge (and the Internet) told me there were many possible “correct” answers. However, only one of these answers would get him full credit on the assignment...More at the link.
In essence, I tried to game the third-grade Pennsylvania System of School Assessment (PSSA), the standardized test for my state. Along with a team of professional developers, I designed artificial-intelligence software to crunch the available data. I talked to teachers. I talked to students. I visited schools and sat through School Reform Commission meetings. After six months of this, I discovered that the test can be gamed. Not by using a beat-the-test strategy, but by a shockingly low-tech strategy: reading the textbook that contains the answers...
This is because standardized tests are not based on general knowledge. As I learned in the course of my investigation, they are based on specific knowledge contained in specific sets of books: the textbooks created by the test makers.
How to do well on standardized school tests
An article at The Atlantic opines that knowledge per se is not the key factor:
