GitHub - dogatuncay/Kmeans-on-Twister-MapReduce: Running Kmeans on Twister MapReduce and finding the best result by using Objective Function

dogatuncay / Kmeans-on-Twister-MapReduce Public

Notifications You must be signed in to change notification settings
Fork 0
Star 1

Running Kmeans on Twister MapReduce and finding the best result by using Objective Function

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
initial_centroid		initial_centroid
result_centroid		result_centroid
results		results
src/cgl/imr/samples/kmeans		src/cgl/imr/samples/kmeans
.gitattributes		.gitattributes
.gitignore		.gitignore
README.txt		README.txt
dtuncay_proj2.jar		dtuncay_proj2.jar
dtuncay_proj2_report.docx		dtuncay_proj2_report.docx
objective_func_result.txt		objective_func_result.txt

Repository files navigation

Distributed kmeans program based on Twister-0.9.
Author: Fei Teng/Doga Tuncay
Data  : 11/9/2011

How to run twister kmeans:

prerequisites:

1. Start apach-activemq-5.4.2

2. Configure and start twister-0.9

3. Create kmeans data directory under $TWISTER_HOME/data

4. run genData script (located in $TWISTER_HOME/sample/kmeans/bin) according to the format:
./gen_data.sh [init clusters file][num of clusters][vector length][sub dir][data file prefix][num of files to generate][num of data points]

5. create partition file (located in $TWISTER_HOME/bin) like this:
./create_partition_file.sh [common directory][file filter][partition file]

6. run kmeansClustering (located in $TWISTER_HOME/samples/kmeans/bin)
./run_kmeans.sh [init clusters file][number of map tasks][partition file]

7. redirect the standard output and collect statistical info.