What is a Data Mining Project?

A data mining project is typically initiated by the business managers or analysts. The purpose of data mining is to identify patterns or trends in large sets of data. For example, a data mining project into customer purchasing trends can help inform the decision making process surrounding the launch of a new product, customer preferences, and priorities.

Every data mining project requires a combination of staff from a range of different areas within the organization. The project manager is usually someone with expertise in business intelligence tools, data mining, and data warehouse support. The subject matter experts are recruited from the different practice areas within the organization. For example, staff from marketing, sales, and accounts receivable would all add value to the project. Information technology staff included in the project would include systems and business analysts.

There are two primary requirements to support a data mining project: software and skill set. There is a wide range of business intelligence software that provides the tools necessary to support data mining. In general, the transactional data needs to be identified and moved into the data warehouse. Once the relational database structure has been created, the data mining tools are used to create custom queries, data cubes, and reports.

Staff working on a data mining project require skills in statistics, information technology, data management, and data integrity issues. The software used in this type of project is very complex, and user training is necessary to utilize the functionality provided. For many organizations, the costs associated with a data mining project must be carefully measured against the possible benefits of this technology.

There are four stages to a data mining project: a requirements document, defining user specifications, implementing the database, and writing queries and reports. The requirements document is created by the project manager, based on discussions with the project sponsor. The purpose of this document is to clearly state the project scope, resources, time line, and delivery date. It is very common to require signatures from the project sponsor and executive level to ensure that senior management has approved the project.

User specifications are often created by a team of business analysts and end users. This process typically requires a series of meetings, review of documentation, and editing. The more collaborative the process, the more satisfactory the end results will be. Working together results in expanded understanding of the users' perspective by the business analysts.

The specifications and requirements document are provided to the business intelligence system team, who isresponsible for creating the database structure, extracting the required data, and working with other team members to set up the infrastructure necessary to support this initiative. This aspect of the project requires time, software, and hardware. In many organizations, a cost benefit analysis is submitted to the project sponsor at this stage, so that funding can be secured.

The actual queries and reports are based on user needs and must be tested by the business analysts before implementation. Look at the core business functions and current reports to see what information is required and determine if data mining is an appropriate tool to meet this need.