MLcomp version alpha Statistics: 1208 users, 401 datasets, 328 programs, 9937 runs (0 queued, 0 running), 0 workers

Run 1808

Creator	pliang
Program	svmlight_multiclass-linear
Dataset	waveform
Task type	MulticlassClassification
Created	1y122d ago
Download	Login required!

Status: Done!

Total Time	46s
Max Memory Usage	37M
Domain:	MulticlassClassification
Learn time
Train error	0.148
Predict train time
Test error	0.143
Predict test time

Log file

===== MAIN: learn based on training data =====
=== START program1: ./run learn ../dataset6/train
=== START program4: ./run split ../dataset6/train ../program1/cv.train ../program1/cv.test
=== END program4: ./run split ../dataset6/train ../program1/cv.train ../program1/cv.test --- OK [1s]
===== Cross-validator: trying hyperparameter 0.01 =====
=== START _tune-hyperparameter0: ./run setHyperparameter 0.01
=== END _tune-hyperparameter0: ./run setHyperparameter 0.01 --- OK [0s]
=== START _tune-hyperparameter0: ./run learn ../cv.train
Using hyperparameter c = 0.01
Reading training examples... (2450 examples) done
Training set properties: 21 features, 3 classes
Iter 1: .........*(NumConst=1, SV=1, CEps=100.0000, QPEps=0.0000)
Iter 2: *(NumConst=2, SV=1, CEps=99.6539, QPEps=0.0000)
Iter 3: .........*(NumConst=3, SV=2, CEps=0.4446, QPEps=0.0000)
Iter 4: .........(NumConst=3, SV=2, CEps=0.0552, QPEps=0.0000)
Final epsilon on KKT-Conditions: 0.05520
Upper bound on duality gap: 0.00055
Dual objective value: dval=0.99944
Primal objective value: pval=0.99999
Total number of constraints in final working set: 3 (of 3)
Number of iterations: 4
Number of calls to 'find_most_violated_constraint': 7350
Number of SV: 2 
Norm of weight vector: |w|=0.03339
Value of slack variable (on working set): xi=99.88853
Value of slack variable (global): xi=99.94373
Norm of longest difference vector: ||Psi(x,y)-Psi(x,ybar)||=5.88275
Runtime in cpu-seconds: 0.02
Final number of constraints in cache: 7343
Compacting linear model...done
Writing learned model...done
=== END _tune-hyperparameter0: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter0: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0
Reading model...done.
Reading test examples... (1050 examples) done.
Classifying test examples...done
Runtime (without IO) in cpu-seconds: 0.00
Average loss on test set: 36.2857
Zero/one-error on test set: 36.29% (669 correct, 381 incorrect, 1050 total)
=== END _tune-hyperparameter0: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0 --- OK [0s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions0 --- OK [0s]
CV error rate 0.362857142857143 with hyperparameter 0.01

===== Cross-validator: trying hyperparameter 0.1 =====
=== START _tune-hyperparameter1: ./run setHyperparameter 0.1
=== END _tune-hyperparameter1: ./run setHyperparameter 0.1 --- OK [0s]
=== START _tune-hyperparameter1: ./run learn ../cv.train
Using hyperparameter c = 0.1
Reading training examples... (2450 examples) done
Training set properties: 21 features, 3 classes
Iter 1: .........*(NumConst=1, SV=1, CEps=100.0000, QPEps=0.0000)
Iter 2: *(NumConst=2, SV=1, CEps=96.5393, QPEps=0.0000)
Iter 3: .........*(NumConst=3, SV=2, CEps=4.4462, QPEps=0.0000)
Iter 4: *(NumConst=4, SV=3, CEps=0.5520, QPEps=0.3807)
Iter 5: .........(NumConst=4, SV=3, CEps=0.0431, QPEps=0.3807)
Final epsilon on KKT-Conditions: 0.38067
Upper bound on duality gap: 0.00441
Dual objective value: dval=9.98489
Primal objective value: pval=9.98929
Total number of constraints in final working set: 4 (of 4)
Number of iterations: 5
Number of calls to 'find_most_violated_constraint': 7350
Number of SV: 3 
Norm of weight vector: |w|=0.17387
Value of slack variable (on working set): xi=99.69868
Value of slack variable (global): xi=99.74178
Norm of longest difference vector: ||Psi(x,y)-Psi(x,ybar)||=5.88275
Runtime in cpu-seconds: 20.77
Final number of constraints in cache: 7343
Compacting linear model...done
Writing learned model...done
=== END _tune-hyperparameter1: ./run learn ../cv.train --- OK [21s]
=== START _tune-hyperparameter1: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1
Reading model...done.
Reading test examples... (1050 examples) done.
Classifying test examples...done
Runtime (without IO) in cpu-seconds: 0.01
Average loss on test set: 38.1905
Zero/one-error on test set: 38.19% (649 correct, 401 incorrect, 1050 total)
=== END _tune-hyperparameter1: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1 --- OK [0s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions1 --- OK [0s]
CV error rate 0.381904761904762 with hyperparameter 0.1

===== Cross-validator: trying hyperparameter 1.0 =====
=== START _tune-hyperparameter2: ./run setHyperparameter 1.0
=== END _tune-hyperparameter2: ./run setHyperparameter 1.0 --- OK [0s]
=== START _tune-hyperparameter2: ./run learn ../cv.train
Using hyperparameter c = 1.0
Reading training examples... (2450 examples) done
Training set properties: 21 features, 3 classes
Iter 1: .........*(NumConst=1, SV=1, CEps=100.0000, QPEps=0.0000)
Iter 2: *(NumConst=2, SV=2, CEps=50.6327, QPEps=14.7605)
Iter 3: .........*(NumConst=3, SV=2, CEps=44.4622, QPEps=0.0001)
Iter 4: *(NumConst=4, SV=3, CEps=5.5202, QPEps=3.8645)
Iter 5: .........*(NumConst=5, SV=2, CEps=0.4588, QPEps=0.0000)
Iter 6: *(NumConst=6, SV=2, CEps=0.1099, QPEps=0.0000)
Iter 7: .........(NumConst=6, SV=2, CEps=0.0260, QPEps=0.0000)
Final epsilon on KKT-Conditions: 0.02598
Upper bound on duality gap: 0.02598
Dual objective value: dval=98.53989
Primal objective value: pval=98.56588
Total number of constraints in final working set: 6 (of 6)
Number of iterations: 7
Number of calls to 'find_most_violated_constraint': 9800
Number of SV: 2 
Norm of weight vector: |w|=1.70886
Value of slack variable (on working set): xi=97.07979
Value of slack variable (global): xi=97.10577
Norm of longest difference vector: ||Psi(x,y)-Psi(x,ybar)||=5.88275
Runtime in cpu-seconds: 24.18
Final number of constraints in cache: 7343
Compacting linear model...done
Writing learned model...done
=== END _tune-hyperparameter2: ./run learn ../cv.train --- OK [24s]
=== START _tune-hyperparameter2: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2
Reading model...done.
Reading test examples... (1050 examples) done.
Classifying test examples...done
Runtime (without IO) in cpu-seconds: 0.00
Average loss on test set: 37.7143
Zero/one-error on test set: 37.71% (654 correct, 396 incorrect, 1050 total)
=== END _tune-hyperparameter2: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2 --- OK [0s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions2 --- OK [0s]
CV error rate 0.377142857142857 with hyperparameter 1.0

===== Cross-validator: trying hyperparameter 10.0 =====
=== START _tune-hyperparameter3: ./run setHyperparameter 10.0
=== END _tune-hyperparameter3: ./run setHyperparameter 10.0 --- OK [0s]
=== START _tune-hyperparameter3: ./run learn ../cv.train
Using hyperparameter c = 10.0
Reading training examples... (2450 examples) done
Training set properties: 21 features, 3 classes
Iter 1: .........*(NumConst=1, SV=1, CEps=100.0000, QPEps=0.0000)
Iter 2: *(NumConst=2, SV=2, CEps=59.2104, QPEps=0.0000)
Iter 3: *(NumConst=3, SV=3, CEps=39.3726, QPEps=0.0000)
Iter 4: .........*(NumConst=4, SV=3, CEps=100.0388, QPEps=0.0001)
Iter 5: *(NumConst=5, SV=3, CEps=17.8524, QPEps=0.0000)
Iter 6: .........*(NumConst=6, SV=4, CEps=38.6092, QPEps=18.1495)
Iter 7: *(NumConst=7, SV=3, CEps=28.8648, QPEps=6.6857)
Iter 8: *(NumConst=8, SV=3, CEps=19.2494, QPEps=2.0337)
Iter 9: *(NumConst=9, SV=4, CEps=4.0995, QPEps=1.5735)
Iter 10: *(NumConst=10, SV=3, CEps=4.8316, QPEps=0.9777)
Iter 11: .........*(NumConst=11, SV=3, CEps=2.6708, QPEps=0.4194)
Iter 12: *(NumConst=12, SV=3, CEps=1.5446, QPEps=0.0047)
Iter 13: *(NumConst=13, SV=3, CEps=0.6333, QPEps=0.0205)
Iter 14: *(NumConst=14, SV=2, CEps=0.3231, QPEps=0.0000)
Iter 15: .........*(NumConst=15, SV=2, CEps=0.1394, QPEps=0.0000)
Iter 16: .........(NumConst=15, SV=2, CEps=0.0832, QPEps=0.0000)
Final epsilon on KKT-Conditions: 0.08323
Upper bound on duality gap: 0.83234
Dual objective value: dval=858.82585
Primal objective value: pval=859.65820
Total number of constraints in final working set: 15 (of 15)
Number of iterations: 16
Number of calls to 'find_most_violated_constraint': 14700
Number of SV: 2 
Norm of weight vector: |w|=15.75177
Value of slack variable (on working set): xi=73.47666
Value of slack variable (global): xi=73.55990
Norm of longest difference vector: ||Psi(x,y)-Psi(x,ybar)||=7.07469
Runtime in cpu-seconds: 0.06
Final number of constraints in cache: 7153
Compacting linear model...done
Writing learned model...done
=== END _tune-hyperparameter3: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter3: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3
Reading model...done.
Reading test examples... (1050 examples) done.
Classifying test examples...done
Runtime (without IO) in cpu-seconds: 0.00
Average loss on test set: 36.3810
Zero/one-error on test set: 36.38% (668 correct, 382 incorrect, 1050 total)
=== END _tune-hyperparameter3: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3 --- OK [0s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions3 --- OK [1s]
CV error rate 0.363809523809524 with hyperparameter 10.0

===== Cross-validator: trying hyperparameter 100.0 =====
=== START _tune-hyperparameter4: ./run setHyperparameter 100.0
=== END _tune-hyperparameter4: ./run setHyperparameter 100.0 --- OK [0s]
=== START _tune-hyperparameter4: ./run learn ../cv.train
Using hyperparameter c = 100.0
Reading training examples... (2450 examples) done
Training set properties: 21 features, 3 classes
Iter 1: .........*(NumConst=1, SV=1, CEps=100.0000, QPEps=0.0000)
Iter 2: *(NumConst=2, SV=2, CEps=59.2104, QPEps=0.0000)
Iter 3: *(NumConst=3, SV=3, CEps=39.3726, QPEps=0.0000)
Iter 4: *(NumConst=4, SV=3, CEps=27.6723, QPEps=0.0000)
Iter 5: *(NumConst=5, SV=5, CEps=15.5168, QPEps=2.0925)
Iter 6: .........*(NumConst=6, SV=6, CEps=85.9361, QPEps=4.8732)
Iter 7: *(NumConst=7, SV=5, CEps=37.4726, QPEps=14.9473)
Iter 8: *(NumConst=8, SV=5, CEps=46.3594, QPEps=18.2630)
Iter 9: *(NumConst=9, SV=5, CEps=50.4887, QPEps=17.9720)
Iter 10: *(NumConst=10, SV=4, CEps=17.5847, QPEps=4.7444)
Iter 11: *(NumConst=11, SV=4, CEps=9.1240, QPEps=3.7676)
Iter 12: *(NumConst=12, SV=6, CEps=12.6638, QPEps=4.5485)
Iter 13: *(NumConst=13, SV=6, CEps=13.6662, QPEps=3.9109)
Iter 14: *(NumConst=14, SV=5, CEps=9.1611, QPEps=4.4258)
Iter 15: .........*(NumConst=15, SV=5, CEps=10.7684, QPEps=5.0403)
Iter 16: *(NumConst=16, SV=5, CEps=6.6054, QPEps=2.9872)
Iter 17: *(NumConst=17, SV=5, CEps=5.7962, QPEps=2.1775)
Iter 18: *(NumConst=18, SV=4, CEps=4.4228, QPEps=1.1505)
Iter 19: *(NumConst=19, SV=4, CEps=1.6699, QPEps=0.2354)
Iter 20: *(NumConst=20, SV=4, CEps=1.6500, QPEps=0.6007)
Iter 21: *(NumConst=21, SV=5, CEps=1.2451, QPEps=0.4743)
Iter 22: .........*(NumConst=22, SV=5, CEps=0.5889, QPEps=0.2623)
Iter 23: *(NumConst=23, SV=5, CEps=0.6205, QPEps=0.2463)
Iter 24: *(NumConst=24, SV=4, CEps=0.5982, QPEps=0.2373)
Iter 25: *(NumConst=25, SV=5, CEps=0.3166, QPEps=0.1457)
Iter 26: *(NumConst=26, SV=4, CEps=0.2647, QPEps=0.1096)
Iter 27: *(NumConst=27, SV=4, CEps=0.1437, QPEps=0.0594)
Iter 28: *(NumConst=28, SV=4, CEps=0.1006, QPEps=0.0486)
Iter 29: .........(NumConst=28, SV=4, CEps=0.0484, QPEps=0.0486)
Final epsilon on KKT-Conditions: 0.04856
Upper bound on duality gap: 4.90519
Dual objective value: dval=5452.43016
Primal objective value: pval=5457.33535
Total number of constraints in final working set: 28 (of 28)
Number of iterations: 29
Number of calls to 'find_most_violated_constraint': 12250
Number of SV: 4 
Norm of weight vector: |w|=48.18459
Value of slack variable (on working set): xi=42.93895
Value of slack variable (global): xi=42.96458
Norm of longest difference vector: ||Psi(x,y)-Psi(x,ybar)||=5.88275
Runtime in cpu-seconds: 0.16
Final number of constraints in cache: 6842
Compacting linear model...done
Writing learned model...done
=== END _tune-hyperparameter4: ./run learn ../cv.train --- OK [0s]
=== START _tune-hyperparameter4: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4
Reading model...done.
Reading test examples... (1050 examples) done.
Classifying test examples...done
Runtime (without IO) in cpu-seconds: 0.00
Average loss on test set: 16.2857
Zero/one-error on test set: 16.29% (879 correct, 171 incorrect, 1050 total)
=== END _tune-hyperparameter4: ./run predict ../cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4 --- OK [0s]
=== START program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4
=== END program5: ./run evaluate ../program1/cv.test /home/mlcomp/worker1/scratch/program1/cvTestPredictions4 --- OK [0s]
CV error rate 0.162857142857143 with hyperparameter 100.0

Best hyperparameter value is 100.0; got CV error rate 0.162857142857143
=== END program1: ./run learn ../dataset6/train --- OK [47s]

===== MAIN: predict/evaluate on train data =====
=== START program7: ./run stripLabels ../dataset6/train ../program0/evalTrain.in
=== END program7: ./run stripLabels ../dataset6/train ../program0/evalTrain.in --- OK [0s]
=== START program1: ./run predict ../program0/evalTrain.in ../program0/evalTrain.out
=== START _tune-hyperparameter-best: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out
Reading model...done.
Reading test examples... (3500 examples) done.
Classifying test examples...done
Runtime (without IO) in cpu-seconds: 0.00
Average loss on test set: 75.0857
Zero/one-error on test set: 75.09% (872 correct, 2628 incorrect, 3500 total)
=== END _tune-hyperparameter-best: ./run predict ../../program0/evalTrain.in ../../program0/evalTrain.out --- OK [0s]
=== END program1: ./run predict ../program0/evalTrain.in ../program0/evalTrain.out --- OK [0s]
=== START program8: ./run evaluate ../dataset6/train ../program0/evalTrain.out
=== END program8: ./run evaluate ../dataset6/train ../program0/evalTrain.out --- OK [1s]

===== MAIN: predict/evaluate on test data =====
=== START program7: ./run stripLabels ../dataset6/test ../program0/evalTest.in
=== END program7: ./run stripLabels ../dataset6/test ../program0/evalTest.in --- OK [0s]
=== START program1: ./run predict ../program0/evalTest.in ../program0/evalTest.out
=== START _tune-hyperparameter-best: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out
Reading model...done.
Reading test examples... (1500 examples) done.
Classifying test examples...done
Runtime (without IO) in cpu-seconds: 0.00
Average loss on test set: 74.9333
Zero/one-error on test set: 74.93% (376 correct, 1124 incorrect, 1500 total)
=== END _tune-hyperparameter-best: ./run predict ../../program0/evalTest.in ../../program0/evalTest.out --- OK [0s]
=== END program1: ./run predict ../program0/evalTest.in ../program0/evalTest.out --- OK [0s]
=== START program8: ./run evaluate ../dataset6/test ../program0/evalTest.out
=== END program8: ./run evaluate ../dataset6/test ../program0/evalTest.out --- OK [0s]


real	0m47.633s
user	0m46.795s
sys	0m0.320s

Run specification

• supervised-learning: Main entry for supervised learning for training and testing a program on a dataset.
  • (learner:Program) tune-hyperparameter: Sets the hyperparameter
    • (numProbes:int) 5
    • (learner:Program) svmlight_multiclass-linear: SVMlight for multiclass classification (http://svmlight.joachims.org/svm_multiclass.html)
    • (splitter:Program) multiclass-utils: Validates and inspects a dataset in MulticlassClassification format.
    • (evaluator:Program[Evaluate]) classification-evaluator: Evaluates predictions of classification datasets (discrete outputs).
  • (dataset:Dataset) waveform: 5000 examples, 21 features
  • (stripper:Program[Strip]) multiclass-utils: Validates and inspects a dataset in MulticlassClassification format.
  • (evaluator:Program[Evaluate]) classification-evaluator: Evaluates predictions of classification datasets (discrete outputs).

Results

• doTest:
  • evaluate:
    • errorRate: 0.143333333333333
    • numErrors: 215
    • numExamples: 1500
    • success: true
    • time: 0
  • predict:
    • predict:
      • success: true
      • time: 0
    • success: true
    • time: 0
  • strip:
    • success: true
    • time: 0
• doTrain:
  • evaluate:
    • errorRate: 0.148285714285714
    • numErrors: 519
    • numExamples: 3500
    • success: true
    • time: 1
  • predict:
    • predict:
      • success: true
      • time: 0
    • success: true
    • time: 0
  • strip:
    • success: true
    • time: 0
• exitCode: 0
• learn:
  • bestCVErrorRate: 0.162857142857143
  • bestHyperparameter: 100.0
  • evaluate0:
    • errorRate: 0.362857142857143
    • numErrors: 381
    • numExamples: 1050
    • success: true
    • time: 0
  • evaluate1:
    • errorRate: 0.381904761904762
    • numErrors: 401
    • numExamples: 1050
    • success: true
    • time: 0
  • evaluate2:
    • errorRate: 0.377142857142857
    • numErrors: 396
    • numExamples: 1050
    • success: true
    • time: 0
  • evaluate3:
    • errorRate: 0.363809523809524
    • numErrors: 382
    • numExamples: 1050
    • success: true
    • time: 1
  • evaluate4:
    • errorRate: 0.162857142857143
    • numErrors: 171
    • numExamples: 1050
    • success: true
    • time: 0
  • learn0:
    • success: true
    • time: 0
  • learn1:
    • success: true
    • time: 21
  • learn2:
    • success: true
    • time: 24
  • learn3:
    • success: true
    • time: 0
  • learn4:
    • success: true
    • time: 0
  • predict0:
    • success: true
    • time: 0
  • predict1:
    • success: true
    • time: 0
  • predict2:
    • success: true
    • time: 0
  • predict3:
    • success: true
    • time: 0
  • predict4:
    • success: true
    • time: 0
  • setHyperparameter0:
    • success: true
    • time: 0
  • setHyperparameter1:
    • success: true
    • time: 0
  • setHyperparameter2:
    • success: true
    • time: 0
  • setHyperparameter3:
    • success: true
    • time: 0
  • setHyperparameter4:
    • success: true
    • time: 0
  • split:
    • success: true
    • time: 1
  • success: true
  • time: 47
• success: true

Comments: