diff --git a/README.md b/README.md index 08d1bfa..756cbd4 100644 --- a/README.md +++ b/README.md @@ -52,7 +52,7 @@ Run.py Parameters: - chembl-1024-jaccard - molport-1024-jaccard algorithm: algorithm name (Required) - Options: + Choices: - Balltree(Sklearn) - Bruteforce - Datasketch @@ -100,6 +100,67 @@ Command Examples (for Singularity only): python run.py --dataset=chembl-1024-jaccard --algorithm='Hnsw(Nmslib)' --count=100 --sif-dir="./singularity" --batch +# Visualization of Execution Results under a PC +1. Run python plot.py + +plot.py Parameters: + + dataset: dataset name (Required) + Examples: + - chembl-1024-jaccard + - molport-1024-jaccard + count: the value of K for top-K nearest neighbor search + Default: 10 + output/-o: the output file + x-axis/-x: which metric to use on the X-axis + Choices: + - k-nn: Recall for top-K nearest neighbor search (Default) + - range: Recall for range query + - qps: Queries per second (1/s) + - build: Indexing time (s) + - indexsize: Index size (kB) + y-axis/-y: which metric to use on the Y-axis + Choices: + - k-nn: Recall for top-K nearest neighbor search + - range: Recall for range query + - qps: Queries per second (1/s) (Default) + - build: Indexing time (s) + - indexsize: Index size (kB) + x-log/-X: Draw the X-axis using a logarithmic scale + Default: False + y-log/-Y: draw the Y-axis using a logarithmic scale + Default: False + raw: show raw results (not just Pareto frontier) in faded colours + Default: False + batch: batch query mode + Default: False + rq: range query / threshold-based query mode + Default: False + radius: in the range query mode, the used cut-off value. Here the distance is used, so if all near neighbors with a similarity coefficient larger than 0.8, please set it 0.2. + Default: 0.3 + + +Command Examples: +- Plot results on chembl-1024-jaccard dataset for top-K (K=100) nearest neighbor query to "results/chembl-1024-jaccard-100.png". X-axis: recall. Y-axis: qps, log-scale. + + python plot.py --dataset=chembl-1024-jaccard -Y --count=100 -o=results/chembl-1024-jaccard-100 + +- Plot results on molport-1024-jaccard dataset for top-K (K=10) nearest neighbor query to "results/molport-1024-jaccard-indexsize-10.png". X-axis: recall. Y-axis: index size, log-scale. + + ython plot.py --dataset=molport-1024-jaccard -Y -y=indexsize --count=10 -o=results/molport-1024-jaccard-indexsize-10 + +- Plot results on molport-1024-jaccard dataset for top-K (K=10) nearest neighbor query to "results/molport-1024-jaccard-buildtime-10.png". X-axis: recall. Y-axis: indexing time, log-scale. + + python plot.py --dataset=molport-1024-jaccard -Y -y=build --count=10 -o=results/molport-1024-jaccard-buildtime-10 + +- Plot batch mode results on molport-1024-jaccard dataset for top-K (K=100) nearest neighbor query to "results/molport-1024-jaccard-batch-100.png". X-axis: recall. Y-axis: qps, log-scale. + + python plot.py --dataset=molport-1024-jaccard -Y --batch --count=100 -o=results/molport-1024-jaccard-batch-100 + +- Plot batch mode results on chembl-1024-jaccard dataset for range query with similarity cutoff 0.6 to "results/chembl-1024-jaccard-0_4.png". X-axis: recall (range query). Y-axis: qps, log-scale. + + python plot.py --dataset=chembl-1024-jaccard -Y -x=range --rq --radius=0.4 -o=results/chembl-1024-jaccard-0_4 + # Executions under an HPC environment 1. Load anaconda module @@ -143,6 +204,34 @@ An example "run.sh": python run.py --dataset=chembl-1024-jaccard --algorithm='Hnsw(Nmslib)' --count=100 --sif-dir="./singularity" +# Visualization of Execution Results under an HPC environment +Run your algorithm scripts by SLURM shell + + sbatch plot.sh + +An example "plot.sh": + + #!/bin/bash + + #SBATCH --ntasks=1 + #SBATCH --exclude=cn[65-69,71-136,325-343,345-353,355-358,360-364,369-398,400-401],gpu[07-10] + + + module load anaconda/5.1.0 + + source activate ann_env + + module purge + + module load gcc/5.4.0 + + module load singularity/3.1 + + + + python plot.py --dataset=chembl-1024-jaccard -Y --count=100 -o=results/chembl-1024-jaccard-100 + + # Parameter tuning All algorithmic parameter settings are included in the "./algos.yaml" file.