Blind Oud Test - Origins

[powdernose]

There was a little discussion regarding blind testing oud in the 'oud news' thread (mostly my posts), but I thought I'd collate the relative posts in a separate thread in hope of giving the idea a fighting chance:

Secondly, with regard to the true value of $2500 oud oils, I think the only way to fairly tackle the subject would be to conduct a blind sample test. Of course this would require men with the means to donate costly samples, and of course the corresponding generosity. It might even require courage to make such a donation, as a 2500$ oil might depreciate in face of such a valuation. Then one would need careful and impartial design of the test by a referee or referees. Dependable testers, a single format for testing and recording impressions, time limits on delivering findings, etc.
However, by mixing a handful of unknown oud oils from different perceived values, and accumulating their scores from completely blind judgments, one could at least get a better snapshot regarding the oils' true olfactory values.

That's it!

The rest will be done by the olfactory. Such tests have been made; therefore, price has been considered the Promethean unbound. Otherwise, every body is sensible enough to understand the rationale behind pricing and other matters, if any.

..

I found my thoughts wandering last night and I kept imagining an oud blind test challenge.
Someone should really get the ball rolling on something like that.
And not just for the reality check regarding perceived value, but also for the pure thrill of such an exercise,
I think ouddicts everywhere would love to observe such a challenge and that beyond the educational process, it should reignite a spark in our shared passion.

@powdernose many claim to be objective....this i feel is the only real way and have said so for a long time! Blind testing no story, info or vendor mention.....pure scent and nose.

Rally the troops!!

Yes, and beyond the obvious biases that vendor, price, story, suggestion and affiliation might inflict,
what people may not realise is just how naked and introspective blind testing can be;
unconscious preconceptions, true olfactory ability, unexpected inclinations, foundation of preference - everything is stripped and challenged.

I agree and it needs someone who likes all types of oud and known to be skilled

In my opinion that would re-introduce some bias.
A good panel should be a rainbow panel, and, I think, should include one 'virgin' nose.
Of course you'd want a couple of skilled of noses in there! And in fact, you'd want the reviewers equipped with fine communicative skills.
But I believe one should also seek experienced ouddicts, regardless of personal oud preferences, and regardless of their command of language.

Great Idea, it would be an experiment to try out perhaps at the Oud fayre. Each vendor can supply 3 or 4 samples labeled 1 to 4 or a third party person could take samples from each vendor and label them without letting anyone know. Then through the course of the weekend visitors can try the oils and give them a numerical rating...? something along these lines would be very interesting

A great start! And it sounds like it would be fun for the fayre.
But what I envisioned was an intensive exercise of testing.
Being at a fayre provides neither the required time to truly test an oud out, nor the necessary solitude to be introspective and analytical.

The challenge is how to do this and be fair?

Precisely!

I will have a think as its not very simple, any suggestions from Ouddicts would be welcome. We could have 3 members opt for the blind testing and pay postage,

I'd suggest 5 testers.
One idea is not just an opt in, but actual selection of candidates by the community.
Another idea I had when I first imagined this, is having secret candidates. You get selected, receive an invitation and opt in or out,
but are not allowed to tell anyone else you have been approached or are participating until results are posted.

i could pick out random oil from say 4 vendors from my collection and send them out. Or vendors could send me samples they want blind tested i would be happy to offer Habzoud oils. We could have a section created on the forum for the blind testing reviews, results.....

That would depend on the extent of your collection and what knowledge any participant has of the content of your collection.
Another of my original thoughts was to have a neutral referee send out samples.
1. The referee receives oils from sources A, B, C...
2. The referee decides which oils will be part of the trial. Not every oil from the initial pool will be used.
3. Each participant receives oils with a different numbering. i.e For Participant1 Borneo 50k will be sample no.1, while for Participant2 it will be sample no.3.
4. Results are posted directly to the forum on agreed date by participants, or to another neutral participant.
5. The referee collates the results.

 

[powdernose]

Dear powdernose, thumb up!

'Virgin nose' suggestion is more than appreciable. This is the best way to avoid the pitfalls that an 'experienced nose' is generally willingly prone to jump into.

I second each post including the one with rules and regulations by respected powdernose.

I like the blind test proposal from @Faizal_p

It removes any potential biasedness towards any vendor - be it positive or negative. I've done something similar in experimental / information economics - sometimes they produce very important and revealing information. Looking forward to the experiment.

If I may, I think it will be better to have some ground rules (std practice) before and during application of any oil - for e.g., washing hands and drying them first, not using any other oils, not rubbing the skin area where the oil is applied, no re-application, not trying to find out the vendor's identity by comparing things from his own stash, etc. Experienced users will not do those things, but for novices it is worth mentioning. The co-ordinators and experts can decide what goes into the list of standard practices. The results will then be even more robust.

Definitely,
as well as a standard format for formulating results.

Seeking to identify a blind sample by hunting for an identical in your own stash is obviously poor practice!
In fact any attempt to identify any sample, by any means, would be foul play. I'd say anyone who can't resist that basic temptation is a bad candidate and should exclude themselves.

Other points you raise seem far less important, but some basic rules could help.

As for novices, my idea was to have a single novice, mostly as means of an equaliser.
I don't think any novice would have any particular difficulty in completing the exercise. No need for 'training'.

Regarding a standard format for reporting results,
I thought I'd draft a Result Sheet for blind testing, as a starting idea, as a springboard.

attached is a screenshot of the draft excel sheet

Loving the ideas @powdernose. Just one point I’d like to make. One thing I have found in my experience is that sometimes one smells an unbranded Oud in isolation - almost akin to a blind test. Immediately an Oud from one of the well known established vendors comes to mind and you think, damn this is so good, so much cheaper and smells like Oud X from Vendor Y.

A few days later you decided to compare the Ouds side by side (a few days as I’m too busy with a 100 other things ).

As you make the comparison, you immediately realise that your olfactory member must be pretty rubbish as Oud X from Vendor Y is far far better than this other oil that evoked memories of Oud X only a couple of days ago. However, side by side it’s a totally different experience and there is little to no similarity.

To me, this is a problem with blind testing, unless a good representative sample of other Ouds are also on hand.

That is a good point.
It has probably happened to all of us. However, the more you go through this process, the more you strengthen your oud experience.
At least I think so.
Someone who has gone through this process over and over again should be more experienced than someone who has not.

Also, if I may counter, you are referring to smelling an oil once in isolation and making associations.
What I'm recommending is intensely testing a cluster of samples. So no oil is ever in total isolation, and you'd be smelling the oils many times before writing any assessment.

No testing method can be perfect

Ah yes... testing a cluster makes sense

Too bad that dogs can’t speak human language, they have much better sense of smell than we do.
It would be interesting to hear their opinion.


Would the estimated age of oud find a place in this result sheet table as well? I read here and there that aging process makes oud oils scent more rounded and smooth.

It could sure, I think one should be open to both additions and subtractions.

In my own view though I would not add an age estimation.
I think it is too much to ask of the poor nose.
I can't see any appreciable benefit to adding it either.

 

[F4R1d0uX]

Nice job

 

[powdernose]

Blind test result sheet draft

https://www.dropbox.com/s/8w7ly8kcm7i6v0k/Blind Oud Test Form.xlsx?dl=0

 

[powdernose]

What this project really needs is generosity, curiosity, and benefactors!

So if you're interested, contact me.

 

[Rai Munir]

Respected powdernose, I think all members will definitely be willing to join this project. At least once we have to materialize it. I have downloaded the sheet, and, in my opinion, it will do.

I think the members are busy in Eid celebrations; otherwise, the thread would have received welcoming feedback.

 

[Ouddict]

@powdernose fantastic idea. Wish more members were like you. Do you think it might be an idea for me to make the list on software so people can add to it online?

 

[Rai Munir]

'A Blind Oud Test' to 'See Better'!

 

[Rai Munir]

@powdernose fantastic idea. Wish more members were like you. Do you think it might be an idea for me to make the list on software so people can add to it online?

Well, the idea requires extreme confidentiality and impartiality. Second, I think it was discussed that samples are to be sent to some newbies with virgin noses and to some veterans. This is what I think is the backbone of the project.

The question is 'How to materialize it?'.
It is possible if Admin and Mods show consent to initiate the project. Members will definitely be ready, but vendors, hmmm, some will be ready, but majority will be reluctant.

But we are here to initiate on our own for learning, and not to let down or step down some respectable vendor.

 

[Louis Miller]

Well, the idea requires extreme confidentiality and impartiality. Second, I think it was discussed that samples are to be sent to some newbies with virgin noses and to some veterans. This is what I think is the backbone of the project.

The question is 'How to materialize it?'.
It is possible if Admin and Mods show consent to initiate the project. Members will definitely be ready, but vendors, hmmm, some will be ready, but majority will be reluctant.

But we are here to initiate on our own for learning, and not to let down or step down some respectable vendor.

This sounds very exciting, I do hope such a test might be possible. Perhaps there could be a participation fee to help offset sample costs? I know I'd be happy to pay for a chance to put my newbie nose to the test.

 

[powdernose]

@powdernose fantastic idea. Wish more members were like you. Do you think it might be an idea for me to make the list on software so people can add to it online?

What kind of list?

Well, the idea requires extreme confidentiality and impartiality. Second, I think it was discussed that samples are to be sent to some newbies with virgin noses and to some veterans. This is what I think is the backbone of the project.

The question is 'How to materialize it?'.
It is possible if Admin and Mods show consent to initiate the project. Members will definitely be ready, but vendors, hmmm, some will be ready, but majority will be reluctant.

But we are here to initiate on our own for learning, and not to let down or step down some respectable vendor.

Confidentiality and impartiality are what we would strive for.

A few materialisation ideas have already been mentioned. And I personally have more ideas.

Vendor participation would be good. Like you say some would be willing,
and only a few would need to be willing in truth.
The second part is having a couple of private collectors that are willing to sacrifice a couple of ml of 'legends'.
Such benefactors would be enough to kickstart this into motion

This sounds very exciting, I do hope such a test might be possible. Perhaps there could be a participation fee to help offset sample costs? I know I'd be happy to pay for a chance to put my newbie nose to the test.

It is an idea yes, but only as an offset I think.

 

[powdernose]

@powdernose fantastic idea. Wish more members were like you. Do you think it might be an idea for me to make the list on software so people can add to it online?

Ahh, the result sheet you mean?
What would be the purpose of open editing though?

 

[Ouddict]

Blind test result sheet draft

https://www.dropbox.com/s/8w7ly8kcm7i6v0k/Blind Oud Test Form.xlsx?dl=0

I’m talking about this sheet... maybe we can put a permanent table online to collate results. I had a similar idea, but not enough time to execute.

 

[powdernose]

I’m talking about this sheet... maybe we can put a permanent table online to collate results. I had a similar idea, but not enough time to execute.

If I'm understanding you, you mean to re-purpose the sheet to create an online review database, possibly an anonymous database?
Sounds good to me.

 

[Ouddict]

If I'm understanding you, you mean to re-purpose the sheet to create an online review database, possibly an anonymous database?
Sounds good to me.

Yes, exactly

 

[powdernose]

What this project really needs is generosity, curiosity, and benefactors!

So if you're interested, contact me.

Bump.

I'll further specify, so this is not just a vague concept.
I'm hoping to find a couple of benefactors to kick start the project.
Basically, I'd need each private collector to send 1-3 oils of varying grades (at least 1 obviously being high grade) at 5x0.1g to be able to conduct the blind test.
I believe I need 1-2 benefactor commitments to be able to proceed further.

So, PM me, and let me know if you'd be willing to donate (free) to this test.
Or, if you'd be open to accepting an amount of cash in exchange for your offerings (preferably less than the actual value of the oil offerings).

If you PM me with your interest, please do not list the identity or true value of the oils. I wish to remain as blind as possible to their identities and values for the time being.
In the message I just need the commitment to donate/sell, the number of oils you can provide, and the amount of cash required to contribute, if required.

Btw, at he moment, I see myself refereeing/coordinating the test.
If anybody objects, please state so now.

thanks

 

[Rai Munir]

Dear powdernose
This 'Blind Test Theory' is the best I ever read here. Its functionality is evident. What I can contribute is a little money because I am not allowed to ship anything liquid abroad.

What about taking up the project by the Forum managers, officially? If possible.

 

[powdernose]

Dear Rai,

Thank you for the support and the offer.
If and when this progresses, I will contact you.
Likewise, I think things need to be in motion for official support to make any difference.

Dear powdernose
This 'Blind Test Theory' is the best I ever read here. Its functionality is evident. What I can contribute is a little money because I am not allowed to ship anything liquid abroad.

What about taking up the project by the Forum managers, officially? If possible.

 

[Rai Munir]

Dear powdernose
Since the day you floated blind test, I tried it at home and involved my friends.

I am sure it is the only way to achieve some solid empirical findings. Moreover, I am sure that majority of distillers, vendors, vendor distillers and distiller vendors won't second it. And some members too. Or at least the result possibly be nullified. Even some will ask 'For what all this blind test?'. So, I suggested that if it is initiated officially by the Forum administrators, it can be a successful venture.

 

[F4R1d0uX]

Moreover, I am sure that majority of distillers, vendors, vendor distillers and distiller vendors won't second it.

Oh ? I'm in and I was even in before @powdernose 's yesterday post.