Post by Jeremy on Apr 15, 2011 6:14:08 GMT -8
Summary
This post seeks examine how we intuitively make decisions as humans and how well that process can be formalized with a domain expert's knowledge to tune the process. I am curious what others think on the topic and would love to get as many responses from as many people as possible. As for what I consider you an expert in, read on to find out.
Background
Here's a little background as to how this came up.
A few years ago I worked for a company that tracked housing market prices and provided comparables (up to five) to real estate appraisers and the like. Before I arrived the owner and employee calculated the best comps based on the owner's years of domain knowledge in the housing market. He would (manually) query his database for rough matches, producing a list of say 25 or 30; he choose from this pool of comparables and narrowed it down to the best five of those, in his opinion. He then added on manually computed aggregate numbers for those five (min, max, and average values for square footage, age, price, etc) and sent those comps and aggregates to the customer.
While working there I saw he did this over and over and so I suggested an improvement to the process by letting me automate it incorporating his expert domain knowledge into the process. So using his highly-tuned domain knowledge I came up with an equation (specifically a linear combination) to describe how well a comparable ranked against a given home, ie what made one comparable better than another if you have to choose from 20 or 30 similar homes?
Using a simple standard equation, made unique and invaluable with his domain knowledge, I created an automated report that
1) ran the same general query to narrow it down to 25 or so.
2) computed the value of each comp with the equation
3) resorted the list of comps by that value
4) created the report using the top five and their aggregate values
5) emailed the report to the appraiser
I was very pleased one day when we had it all working on my test machine and I ran it against a particular house, creating the report. I then had him and his coworkers do the same report manually and each pick their top five. We then compared each of their intuitively divined results with the automated report. It was very accurate and would pick, in almost every instance, the same comps as they would pick themselves on their own (in of course a fraction of the time). In the times when it didn't pick one or missed one: drilling into the issue, we found that there was some factor the owner was considering that the equation was not. When we redid the equation to consider the new factor it lined up again with what he would have picked manually. It even brought to light some homes which they did not consider and made them go back to see why they didn't pick that one for whatever reason. Anyway, I was actually very proud of this whole thing *small smile*. Better marketing may have made it more of a success. (If you want the link I'll give it to you but be warned it's not Web 2.0; it's rather archaic).
Proposal
The purpose of my post here is to propose a theory that this process is potentially applicable and likely reasonable accurate in other areas of human decision making where qualities and importances are given such that qualities can be measured/ranked with a scalar/numeric value and that both the qualities chosen and weights assigned are given by a domain expert.
So rather than use it to make decisions about how much a house is worth for buying or selling purposes, I would wager it is true in another completely different yet also important decision in life: choosing a mate.
Let me formalize the problem and then I can explain what expert domain knowledge you can give.
It basically works the same way as the homes, you have 20 (give or take a few) people you're considering - probably not 20 all at once
You rank each person on a scale of say 0 to 10 for each quality you consider important: 0 meaning they don't have the quality and 10 being they have the maximum amount that any person can have for that quality.
Say one of the qualities/traits is sense of humor. For example, a score of 1 for sense of humor would mean the person has no sense of humor, ie your local DMV counter person, where a score of 10 is someone who always succeeds in making you laugh, ie, a real Seinfeld or fill in the blank if you don't like Seinfeld.
For the purposes of this, we want to have qualities/traits that can be ranked on a scale of say 0 to 10. So for the requirement that someone be of the same faith it should be stated in terms of spiritual health rather than "is a Christian" (or Buddhist, or atheist, etc) and ranked from 0 - no spiritual life to 10 the best spiritual health anyone can have.
Here's the general equation and formal description (this is the only abstract looking part of this whole post; everything else is concrete):
R(P) = r1*w1 + r2*w2 + r3*w3 + ... + rn*wN
where,
R(P) represents the rank of person P,
ri represents the rating (from 0 to 10) in quality i you assigned for person P and,
wi represents the weight you assigned to quality i.
It does NOT matter whether you understand this piece or not; it's simply to show how it works for those curious to know.
You can still provide feedback w/o having to grasp this.
Of course if you can validate how well your chosen qualities/importances used in the equation for potential partners lined up with your actual real world decision (or soon to be decision).
Example
Here's an example of how it would work. Say a woman is picking between three guys (Think being in the mind of The Bachelorette for instance).
This woman decides on three qualities/traits that are important and she weights
them accordingly. She selects the following:
In picking these qualities and importances she's saying several things. She says that things such as physical health and financial stability are irrelevant. Any two men having exactly the same rating in all three categories (maturity,humor,cooking) where one of them is more financial stable and in perfect health and the other who is unemployed and has cancer will not make the employed healthy man appear as the better choice; they will be considered equal. If financial stability is important even if it's only a little important to her she should list it. The point being, in giving your answer make sure everything important to you is listed.
Here she says that cooking ability is five (5) times more important that spiritual maturity. Something's not quite right with her priorities. But those are her choices so let's see how these three guys she's seeing rank:
Here's how she ranks them (the ri's):
If we plug in the numbers (keeping in mind the weights this girl chose earlier), we get, assuming my math skills hold up:
R(P) = r1*w1 + r2*w2 + r3*w3
R(Larry) = 10*2 + 10*4 + 2*10 = 80
R(Curly) = 4*2 + 8*4 + 10*10 = 140
R(Moe) = 7*2 + 0*4 + 6*10 = 74
Curly ranks the best choice at 140, followed by Larry and closely behind is Moe.
A more mature woman might swap the weights assigned for cooking ability and spiritual maturity such that they value maturity five times more than cooking ability, in which case Larry would be the best choice (and poor Moe, in last place for either girl).
R(Larry) = 10*10 + 10*4 + 2*2 = 144
R(Curly) = 4*10 + 8*4 + 10*2 = 92
R(Moe) = 7*10 + 0*4 + 6*2 = 82
In my asking here, I'm also making the assumption you picked from multiple people for your spouse/gf/bf. Even if you didn't pick from multiple people, you can still make up some numbers and try it out with some made up people. And give your impressions on how accurate it is.
What I'd like to request from you
Reproduce the table below exactly and just redo the weights for each category so they match your preferences. All qualities here are of equal importance (1). Change them to match your preferences. It'd be cool to do a group aggregate of everyone's values but it only works if everyone does this one at a bare minimum. If you did the calculations for actual people, tell how accurate the equation was compared to your real world intuitive choice (or future choice). Discuss the reasons for your weights.
Also if you have any assumptions such as, the
person must be physically healthy or they must be financially responsible for me to even consider ranking them on a finer
scale such as this equation, state your assumptions explicity so we're all on the same page.
On second thought AFTER writing all this, it may be easier to come up with weights as percentages that total 100%. For example, confidence is worth 10% while maturity is 35% and the others make up the rest of the 100%. Might be easier to visualize.
* Qualities taken from www.vitrolenta.com/relationship/essential-qualities-for-a-soul-mate.html
Roll your own (Optional)
Create your own table using your own qualities and assign weights for your qualities in accordance with your personal preferences; it's completely up to you what you pick, how many you pick, etc. If you did the calculations for actual people, tell how accurate the equation was compared to your real world intuitive choice (or future choice). Discuss the reasons for your qualities and weights. Also if you have any assumptions such as, the
person must be physically healthy or they must be financially responsible for me to even consider ranking them on a finer
scale such as this equation, state your assumptions explicity so we're all on the same page.
*I hope the density of this post doesn't scare people away*
This post seeks examine how we intuitively make decisions as humans and how well that process can be formalized with a domain expert's knowledge to tune the process. I am curious what others think on the topic and would love to get as many responses from as many people as possible. As for what I consider you an expert in, read on to find out.
Background
Here's a little background as to how this came up.
A few years ago I worked for a company that tracked housing market prices and provided comparables (up to five) to real estate appraisers and the like. Before I arrived the owner and employee calculated the best comps based on the owner's years of domain knowledge in the housing market. He would (manually) query his database for rough matches, producing a list of say 25 or 30; he choose from this pool of comparables and narrowed it down to the best five of those, in his opinion. He then added on manually computed aggregate numbers for those five (min, max, and average values for square footage, age, price, etc) and sent those comps and aggregates to the customer.
While working there I saw he did this over and over and so I suggested an improvement to the process by letting me automate it incorporating his expert domain knowledge into the process. So using his highly-tuned domain knowledge I came up with an equation (specifically a linear combination) to describe how well a comparable ranked against a given home, ie what made one comparable better than another if you have to choose from 20 or 30 similar homes?
Using a simple standard equation, made unique and invaluable with his domain knowledge, I created an automated report that
1) ran the same general query to narrow it down to 25 or so.
2) computed the value of each comp with the equation
3) resorted the list of comps by that value
4) created the report using the top five and their aggregate values
5) emailed the report to the appraiser
I was very pleased one day when we had it all working on my test machine and I ran it against a particular house, creating the report. I then had him and his coworkers do the same report manually and each pick their top five. We then compared each of their intuitively divined results with the automated report. It was very accurate and would pick, in almost every instance, the same comps as they would pick themselves on their own (in of course a fraction of the time). In the times when it didn't pick one or missed one: drilling into the issue, we found that there was some factor the owner was considering that the equation was not. When we redid the equation to consider the new factor it lined up again with what he would have picked manually. It even brought to light some homes which they did not consider and made them go back to see why they didn't pick that one for whatever reason. Anyway, I was actually very proud of this whole thing *small smile*. Better marketing may have made it more of a success. (If you want the link I'll give it to you but be warned it's not Web 2.0; it's rather archaic).
Proposal
The purpose of my post here is to propose a theory that this process is potentially applicable and likely reasonable accurate in other areas of human decision making where qualities and importances are given such that qualities can be measured/ranked with a scalar/numeric value and that both the qualities chosen and weights assigned are given by a domain expert.
So rather than use it to make decisions about how much a house is worth for buying or selling purposes, I would wager it is true in another completely different yet also important decision in life: choosing a mate.
Let me formalize the problem and then I can explain what expert domain knowledge you can give.
It basically works the same way as the homes, you have 20 (give or take a few) people you're considering - probably not 20 all at once
You rank each person on a scale of say 0 to 10 for each quality you consider important: 0 meaning they don't have the quality and 10 being they have the maximum amount that any person can have for that quality.Say one of the qualities/traits is sense of humor. For example, a score of 1 for sense of humor would mean the person has no sense of humor, ie your local DMV counter person, where a score of 10 is someone who always succeeds in making you laugh, ie, a real Seinfeld or fill in the blank if you don't like Seinfeld.
For the purposes of this, we want to have qualities/traits that can be ranked on a scale of say 0 to 10. So for the requirement that someone be of the same faith it should be stated in terms of spiritual health rather than "is a Christian" (or Buddhist, or atheist, etc) and ranked from 0 - no spiritual life to 10 the best spiritual health anyone can have.
Here's the general equation and formal description (this is the only abstract looking part of this whole post; everything else is concrete):
R(P) = r1*w1 + r2*w2 + r3*w3 + ... + rn*wN
where,
R(P) represents the rank of person P,
ri represents the rating (from 0 to 10) in quality i you assigned for person P and,
wi represents the weight you assigned to quality i.
It does NOT matter whether you understand this piece or not; it's simply to show how it works for those curious to know.
You can still provide feedback w/o having to grasp this.
Of course if you can validate how well your chosen qualities/importances used in the equation for potential partners lined up with your actual real world decision (or soon to be decision).
Example
Here's an example of how it would work. Say a woman is picking between three guys (Think being in the mind of The Bachelorette for instance).
This woman decides on three qualities/traits that are important and she weights
them accordingly. She selects the following:
| quality | weight(w) |
| spiritual maturity | 2 |
| sense of humor | 4 |
| cooking ability | 10 |
In picking these qualities and importances she's saying several things. She says that things such as physical health and financial stability are irrelevant. Any two men having exactly the same rating in all three categories (maturity,humor,cooking) where one of them is more financial stable and in perfect health and the other who is unemployed and has cancer will not make the employed healthy man appear as the better choice; they will be considered equal. If financial stability is important even if it's only a little important to her she should list it. The point being, in giving your answer make sure everything important to you is listed.
Here she says that cooking ability is five (5) times more important that spiritual maturity. Something's not quite right with her priorities. But those are her choices so let's see how these three guys she's seeing rank:
Here's how she ranks them (the ri's):
| Man | maturity(w=2) | humor(w=4) | cooking(w=10) |
| Larry | 10 | 10 | 2 |
| Curly | 4 | 8 | 10 |
| Moe | 7 | 0 | 6 |
If we plug in the numbers (keeping in mind the weights this girl chose earlier), we get, assuming my math skills hold up:
R(P) = r1*w1 + r2*w2 + r3*w3
R(Larry) = 10*2 + 10*4 + 2*10 = 80
R(Curly) = 4*2 + 8*4 + 10*10 = 140
R(Moe) = 7*2 + 0*4 + 6*10 = 74
Curly ranks the best choice at 140, followed by Larry and closely behind is Moe.
A more mature woman might swap the weights assigned for cooking ability and spiritual maturity such that they value maturity five times more than cooking ability, in which case Larry would be the best choice (and poor Moe, in last place for either girl).
R(Larry) = 10*10 + 10*4 + 2*2 = 144
R(Curly) = 4*10 + 8*4 + 10*2 = 92
R(Moe) = 7*10 + 0*4 + 6*2 = 82
In my asking here, I'm also making the assumption you picked from multiple people for your spouse/gf/bf. Even if you didn't pick from multiple people, you can still make up some numbers and try it out with some made up people. And give your impressions on how accurate it is.
What I'd like to request from you
Reproduce the table below exactly and just redo the weights for each category so they match your preferences. All qualities here are of equal importance (1). Change them to match your preferences. It'd be cool to do a group aggregate of everyone's values but it only works if everyone does this one at a bare minimum. If you did the calculations for actual people, tell how accurate the equation was compared to your real world intuitive choice (or future choice). Discuss the reasons for your weights.
Also if you have any assumptions such as, the
person must be physically healthy or they must be financially responsible for me to even consider ranking them on a finer
scale such as this equation, state your assumptions explicity so we're all on the same page.
| quality | weight(w) |
| Values | 1 |
| Emotional health and maturity | 1 |
| Financial responsibility | 1 |
| Physical appearance and fitness | 1 |
| Communication skills | 1 |
| Confidence | 1 |
On second thought AFTER writing all this, it may be easier to come up with weights as percentages that total 100%. For example, confidence is worth 10% while maturity is 35% and the others make up the rest of the 100%. Might be easier to visualize.
* Qualities taken from www.vitrolenta.com/relationship/essential-qualities-for-a-soul-mate.html
Roll your own (Optional)
Create your own table using your own qualities and assign weights for your qualities in accordance with your personal preferences; it's completely up to you what you pick, how many you pick, etc. If you did the calculations for actual people, tell how accurate the equation was compared to your real world intuitive choice (or future choice). Discuss the reasons for your qualities and weights. Also if you have any assumptions such as, the
person must be physically healthy or they must be financially responsible for me to even consider ranking them on a finer
scale such as this equation, state your assumptions explicity so we're all on the same page.
*I hope the density of this post doesn't scare people away*
OK, so that's not a good reason, but it's true.