What is Data Mining?

Michael Anissimov
Michael Anissimov

Data mining uses a relatively large amount of computing power operating on a large set of data to determine regularities and connections between data points. Algorithms that employ techniques from statistics, machine learning and pattern recognition are used to search large databases automatically. Data mining is also known as Knowledge-Discovery in Databases (KDD).

The information collected during data mining is often presented as charts or graphs.
The information collected during data mining is often presented as charts or graphs.

Like the term artificial intelligence, data mining is an umbrella term that can be applied to a number of varying activities. In the corporate world, data mining is used most frequently to determine the direction of trends and predict the future. It is employed to build models and decision support systems that give people information they can use. Data mining takes a front-line role in the battle against terrorism. It was supposedly used to determine the leader of the 9/11 attacks.

Spam filtering can be considered a form of data mining.
Spam filtering can be considered a form of data mining.

Data miners are statisticians who use techniques with names like near-neighbor models, k-means clustering, holdout method, k-fold cross validation, the leave-one-out method, and so on. Regression techniques are used to subtract irrelevant patterns, leaving only useful information. The term Bayesian is seen frequently in the field, referring to a class of inference techniques that predict the likelihood of future events by combining prior probabilities and probabilities based on conditional events. Spam filtering is arguably a form of data mining, which automatically brings relevant messages to the surface from a chaotic sea of phishing attempts and Viagra pitches.

Decision trees are used to filter mountains of data. In a decision tree, all data passes through an entrance node, where it faces a filter that separates the data into streams depending on its characteristics. For example, data about consumer behavior is likely to be filtered based on demographic factors. Data mining is not primarily about fancy graphs and visualization techniques, but it does employ them to show what it has found. It is known that we can absorb more statistical information visually than verbally and this format for presentation can be very persuasive and powerful if used in the right context.

As our civilization becomes increasingly data-saturated and sensors are distributed en masse into our local environments, we will inadvertently discover things that might be missed on the first pass over. Data mining will let us correct these mistakes and discover new insights based on past data, giving us more bang for our data storage buck.

Michael Anissimov
Michael Anissimov

Michael is a longtime wiseGEEK contributor who specializes in topics relating to paleontology, physics, biology, astronomy, chemistry, and futurism. In addition to being an avid blogger, Michael is particularly passionate about stem cell research, regenerative medicine, and life extension therapies. He has also worked for the Methuselah Foundation, the Singularity Institute for Artificial Intelligence, and the Lifeboat Foundation.

You might also Like

Readers Also Love

Discussion Comments

anon163097

Data mining helped find the leader of the 9/11 attacks? They found George Bush? 9/11 was an inside job.

recapitulate

While data mining can refer to the act of simply trying to organize information, it more likely is a word for the other use mentioned here, when groups like social networks, companies, and other advertisers manage to acquire information about possible customers so that they can tailor ads to them and otherwise inundate consumers with more means of consumption. While perhaps more helpful than getting whatever ads they want to throw at people, it still is an invasion of privacy.

massoud

an exigent question :

what is the difference between 'K' and 'k' or 'B' and 'b' and 'disk' and 'disc' ?

Post your comments
Login:
Forgot password?
Register: