Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not getting cliques for two classifier which are not significantly different #7

Open
BINAYKUMAR943 opened this issue Dec 14, 2021 · 1 comment

Comments

@BINAYKUMAR943
Copy link

BINAYKUMAR943 commented Dec 14, 2021

Below is the result when I am trying to compare four different classifiers.

['CoEye' 'TSF' 'cBOSS' 'Mr-SEQL']
CoEye 5.0
Mr-SEQL 61.0
TSF 8.0
cBOSS 10.0
dtype: float64
CoEye 3.279570
TSF 2.559140
cBOSS 2.537634
Mr-SEQL 1.623656
dtype: float64
('CoEye', 'Mr-SEQL', 1.6071230220582003e-11, True)
('Mr-SEQL', 'TSF', 1.5905280411437273e-07, True)
('Mr-SEQL', 'cBOSS', 1.3264426790567074e-06, True)
('CoEye', 'TSF', 1.6515911035945563e-05, True)
('CoEye', 'cBOSS', 4.231130849948383e-05, True)
('TSF', 'cBOSS', 0.4035623277672452, False)
Index(['CoEye', 'TSF', 'cBOSS', 'Mr-SEQL'], dtype='object')
[1, 2]

From the above result, it is clear that classifiers TSF and cBOSS are not significantly different. But I am not getting any clique for them in the CD.
Below is the CD diagram obtained.

cd-diagram

@hfawaz
Copy link
Owner

hfawaz commented Dec 15, 2021

I am not quite sure, but can you try adding more classifiers since 4 is not enough maybe ?
Since the test is also based on friedmann

Due to the assumption that the test statistic has a chi squared distribution, the p-value is only reliable for n > 10 and more than 6 repeated measurements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants