Individual Contribution to Group Work (Help Needed)

For discussing anything related to physics, biology, chemistry, mathematics, and their practical applications.

Moderator: Flannel Jesus

Re: Individual Contribution to Group Work (Help Needed)

Postby phyllo » Thu Jan 14, 2016 9:39 pm

But that's just noise. Since we'll need to be doing statistical inference from incomplete sets of group scores, the noise will just affect how many projects we'll need to get a statistically significant result.
You don't know what is signal and what is noise because you don't have enough information about the nature of the relationships. You haven't studied the most critical aspect - how does teamwork alter performance?
I'm making specific assumptions here, because of the theory we built into the model. Given a different theory, we could use different assumptions with the same result. However we set up individual ability to contribute to group score, with enough data points we can extract individual contribution to a statistically significant degree (if we have the right theory).
You don't have a theory.

When I asked how ability and teamwork ability affect the group score, you said that you did not know.

You produced a program that arbitrarily assumes a fixed contribution value for each individual in each project. You have no grounds for doing that.

You seem to think that if you get enough data, you will be able to work backwards and calculate how individual contributions produce a group result. The relationships are almost certainly too complex to do that. You're just going to end up with lots of data. You need to start at a more fundamental level, build a theory and test the theory. You need to look at the performance of teams first and only after you understand it, then you can calculate backwards to get individual scores from a group score. (Although the problem may still be unsolvable, you should have a good idea about why it is unsolvable.)
"Who loves not wine, woman and song, remains a fool his whole life long."

"Only the educated are free" - Epictetus
"Music is a higher revelation than all wisdom and philosophy" -Beethoven
"Everyday life is the way" -Wumen
"Do not permit the events of your daily life to bind you, but never withdraw yourself from them" - Wumen
phyllo
ILP Legend
 
Posts: 10110
Joined: Thu Dec 16, 2010 1:41 am
Location: ->.

Re: Individual Contribution to Group Work (Help Needed)

Postby James S Saint » Fri Jan 15, 2016 1:10 am

Carleas wrote:You can approximate the variable, though, using the unwise and demonic tools of statistics. And again, in practice we don't have great information about how e.g. SAT scores relate to human worth, so there are always a lot of unknowns, and we're always just approximating.

Oh I see. So with current information and since everything is just a guess anyway, my best guess is that it is better to just kill off ALL Jews and Blacks, therefore....



Gyahd... :icon-rolleyes:

Sometimes I have to think that homosapian is just too fucking dumb to be a species.
Clarify, Verify, Instill, and Reinforce the Perception of Hopes and Threats unto Anentropic Harmony :)
Else
From THIS age of sleep, Homo-sapien shall never awake.

The Wise gather together to help one another in EVERY aspect of living.

You are always more insecure than you think, just not by what you think.
The only absolute certainty is formed by the absolute lack of alternatives.
It is not merely "do what works", but "to accomplish what purpose in what time frame at what cost".
As long as the authority is secretive, the population will be subjugated.

Amid the lack of certainty, put faith in the wiser to believe.
Devil's Motto: Make it look good, safe, innocent, and wise.. until it is too late to choose otherwise.

The Real God ≡ The reason/cause for the Universe being what it is = "The situation cannot be what it is and also remain as it is".
.
James S Saint
ILP Legend
 
Posts: 25768
Joined: Sun Apr 18, 2010 8:05 pm

Re: Individual Contribution to Group Work (Help Needed)

Postby phyllo » Fri Jan 15, 2016 2:37 pm

One can picture it already...

Two students work on a project and it gets a grade of 75%.

"We have analyzed the data and 65% fits our models better. The other 10% must have been noise. The correct grade is 65%"

"We have also analyzed all your projects and determined what must have been your contributions. Your contribution merits a 63% and yours merits a 68%"

How do you know that the project grade was inappropriate without looking into the details of this project?
How do you know how much effort each of us put into this particular project without looking into the details of this project?

"We have lots of data. We have a model. Software does not lie. You have been assigned your correct grades."
:shock: :o

:D
"Who loves not wine, woman and song, remains a fool his whole life long."

"Only the educated are free" - Epictetus
"Music is a higher revelation than all wisdom and philosophy" -Beethoven
"Everyday life is the way" -Wumen
"Do not permit the events of your daily life to bind you, but never withdraw yourself from them" - Wumen
phyllo
ILP Legend
 
Posts: 10110
Joined: Thu Dec 16, 2010 1:41 am
Location: ->.

Re: Individual Contribution to Group Work (Help Needed)

Postby James S Saint » Fri Jan 15, 2016 2:44 pm

Trying to add resolution to an existing picture only helps you to see what you were expecting or desiring to see. You connect the dots and fill in between the lines with shades of preference.
Clarify, Verify, Instill, and Reinforce the Perception of Hopes and Threats unto Anentropic Harmony :)
Else
From THIS age of sleep, Homo-sapien shall never awake.

The Wise gather together to help one another in EVERY aspect of living.

You are always more insecure than you think, just not by what you think.
The only absolute certainty is formed by the absolute lack of alternatives.
It is not merely "do what works", but "to accomplish what purpose in what time frame at what cost".
As long as the authority is secretive, the population will be subjugated.

Amid the lack of certainty, put faith in the wiser to believe.
Devil's Motto: Make it look good, safe, innocent, and wise.. until it is too late to choose otherwise.

The Real God ≡ The reason/cause for the Universe being what it is = "The situation cannot be what it is and also remain as it is".
.
James S Saint
ILP Legend
 
Posts: 25768
Joined: Sun Apr 18, 2010 8:05 pm

Re: Individual Contribution to Group Work (Help Needed)

Postby phyllo » Fri Jan 15, 2016 2:56 pm

James S Saint wrote:Trying to add resolution to an existing picture only helps you to see what you were expecting or desiring to see. You connect the dots and fill in between the lines with shades of preference.
There was a lot of detail present during the creation of the project but most of it was not recorded. The final mark is a gross sum of the effort. Now he wants to use it to rebuild the details. :confusion-shrug:
"Who loves not wine, woman and song, remains a fool his whole life long."

"Only the educated are free" - Epictetus
"Music is a higher revelation than all wisdom and philosophy" -Beethoven
"Everyday life is the way" -Wumen
"Do not permit the events of your daily life to bind you, but never withdraw yourself from them" - Wumen
phyllo
ILP Legend
 
Posts: 10110
Joined: Thu Dec 16, 2010 1:41 am
Location: ->.

Re: Individual Contribution to Group Work (Help Needed)

Postby James S Saint » Fri Jan 15, 2016 3:33 pm

phyllo wrote:
James S Saint wrote:Trying to add resolution to an existing picture only helps you to see what you were expecting or desiring to see. You connect the dots and fill in between the lines with shades of preference.
There was a lot of detail present during the creation of the project but most of it was not recorded. The final mark is a gross sum of the effort. Now he wants to use it to rebuild the details. :confusion-shrug:

Exactly, like getting a thumb picture of something that you liked and trying to expand it into a full size pristine photo (or trying to modify SCTV cam shots to show fine details .. even if they are often false).

"It doesn't matter if every convicted person was guilty as long as the probability and majority were guilty."
Clarify, Verify, Instill, and Reinforce the Perception of Hopes and Threats unto Anentropic Harmony :)
Else
From THIS age of sleep, Homo-sapien shall never awake.

The Wise gather together to help one another in EVERY aspect of living.

You are always more insecure than you think, just not by what you think.
The only absolute certainty is formed by the absolute lack of alternatives.
It is not merely "do what works", but "to accomplish what purpose in what time frame at what cost".
As long as the authority is secretive, the population will be subjugated.

Amid the lack of certainty, put faith in the wiser to believe.
Devil's Motto: Make it look good, safe, innocent, and wise.. until it is too late to choose otherwise.

The Real God ≡ The reason/cause for the Universe being what it is = "The situation cannot be what it is and also remain as it is".
.
James S Saint
ILP Legend
 
Posts: 25768
Joined: Sun Apr 18, 2010 8:05 pm

Re: Individual Contribution to Group Work (Help Needed)

Postby Moreno » Fri Jan 15, 2016 4:25 pm

Carleas wrote:The projects presented to groups will have similar problems with implicit biases as do standardized tests
Different biases, not the same ones. My point was that thinking that testing somehow removes bias is not correct. You raised the issue of biases. I pointed out that there are biases with testing.
and the format similarly benefits specific traits that we might not want to be testing for e.g. extroversion and physical attractiveness.
Extroversion, I think could be worked against, but making the groups task oriented and with teacher intervention. Not eliminated, but worked against. Of course someone could argue that these same traits match biases in the work world, so the evaluations will be good markers for future success and also attractiveness to employers. I wouldn't want to run that line, but I think then we need to eliminate the conception of eduction as the production of good worker bees.


And the two biases don't seem opposed; there's no reason to think that subjective evaluation will cancel out the bias in the project design. Anonymous grading removes a bias, and leaves other biases untouched.
I think anonymous grading of tests is fine. I am not arguing against it. It just does not deal with the biases of testing as a whole.


I'd be interested to know your thoughts on evaluation. In an ideal world, we might prefer not to evaluate people at all, or at least not so reductively, but in practice reductive evaluation is needed to communicate ability and potential to people who might not be able to observe intangible qualities, or who might need to review so many applicants that a holistic evaluation would be impossible. In those cases, numeric evaluation seems necessary, even in an otherwise student-centered environment.
Well, in a society that is rushing out of control into the future we do have little time to evaluate and so we measure. I can't be critical of one piece of the whole without being critical of the whole. I think extending apprenticeships more generally into society would be good thing. And I do not mean following the old guild model. I think students should be able to apprentice without committing, certainly when the are young. I live somewhere where everything is education/'merit' based. All security guards have gone through exhaustive eductions. Even cleaners and other basic skill jobs are getting educations. There is little sense that most jobs can be learned by smart enough people and that most of the main learning will take place on the job. I would like to see the opposite trend. Where character, determination, personal sense of connection for a core for someone getting into different kinds of internship/apprentice situations that lead to long term placements. Some professions need some real focus on very specific skills and there I can see dealing with grades in number form more. I think engineers could be given problem based (real life scenario problems) tests and get grades on their math etc. and the numbers would be valuable. But how well one does in many jobs is just not going to be reflected by how well you dealt with sitting in rows, cramming for tests, and being, in general a passive intaker of knowledge. I believe there have been some studies that have shown that determination, ability to deal with frustration and obstacles and passion are much better markers for success in most fields. The gifted often end up not shining. And schooling trains a certain kind of relationship to learning and dealing with obstacles/problems that is not so life like. At the very least I think evaluations should be there in addition. And really it is not so hard to dip into even a rather big pile of evaluations of a person and learn much more about them in not much more time. Some businesses actually put people in group problem solving situations with other candidates and watch, and these companies are aware that introverts can be valuable members of a group, so they are not 'fooled' by the flash of the extroverts. Teachers and students could be taught this also. I mean, think about it.

If we are in school to learn and we just assume that students cannot learn about biases based on physical attractiveness and extroversion, then we have cut the world up, very oddly, into what is prioritized to be learned (facts and formulas) and wisdom, which is considered off the table. It might be a good idea to work on teaching wisdom and slow the whole thing down.

And by the way, I am not saying that we should have a kind of big brother education where we train people to not have biases, but there are ways to keep the light on such issues, withing jamming conclusions into people's brains, such that people can come to see things in a more complex way.

Tests on the other hand treat everyone AS IF the best knower of history or the user of English or the user of math has the precise same ideal. They should be able to answer more questions correctly than the others.

In a workplace or group of any kind, very different kinds of learners, with very different kinds of skills and personalities can contribute and complexly formed groups do better than groups with, say, all extroverts. If testing is the measure, yes, introverts will not be biased against, though perhaps extroverts will. Some of them at least will tend to make more detail errors, on the other hand they may set the ball rolling and help groups towards solutions faster. This may not be the case about extroverts, but there are going to be character types who will do less well on tests, but add at least as much to professional or educational groups. And they may well flourish in apprentice type situations.

some people like to deal with things as abstracted out problems. Others cannot invest energy and focus until it is part of a project that is real. A lot of the people who get labelled ADHD do better when physical action is part of the learning and when actual projects are taking place. Tests, which they consider more boring than other students do, are severely biased against them.

I've gone a bit random and wandered.
User avatar
Moreno
ILP Legend
 
Posts: 10305
Joined: Sat Nov 24, 2007 5:46 pm

Re: Individual Contribution to Group Work (Help Needed)

Postby pilgrim-seeker_tom » Sat Jan 16, 2016 4:20 pm

Why am I jumping into this OP? ... don't know ... feels like I'm going out on a limb here.

In another forum I suggested that intuition has a role to play in the learning process ... seems to me our education system ... and culture ... suppresses our in born intuition capabilities.

Babies have intuition ... I often make eye contact with babies/toddlers(strangers) ... sometimes I get a smile ... sometimes a frown ... sometimes a scream ... and so on. Seems to me there is an intuitive assessment of compatibility from even brief eye contact ... without any verbal communication.

How about introducing 3 projects to a class ... try to elicit some discussion of what is expected ... and then allow the children to decide which project they want to join ... by secret ballot ... to avoid the natural group associations already existing in the class.

May take a couple of iterations to work out the wrinkles ... results may be surprising ... and then again maybe not so much.
"Do not be influenced by the importance of the writer, and whether his learning be great or small; but let the love of pure truth draw you to read. Do not inquire, “Who said this?” but pay attention to what is said”

Thomas Kempis 1380-1471
User avatar
pilgrim-seeker_tom
Philosopher
 
Posts: 1639
Joined: Sun Dec 17, 2006 11:16 am

Re: Individual Contribution to Group Work (Help Needed)

Postby Carleas » Tue Jan 19, 2016 7:38 pm

phyllo wrote:You haven't studied the most critical aspect - how does teamwork alter performance?

There are at least two ways the idea could go. One is to continue to insist that the best way have to test individual ability is tests like the SAT and an average individual-project scores (GPA). In that case, the best we could do is to correlate group performance to those individual indices. I suspect there would be a strong correlation, but there needn't be. This is an empirical question, and I don't have the data.

Another, the one I'd prefer, is to say that the SAT and individual-project scores aren't especially relevant to what we're really interested in, that in fact teamwork is performance, and so measuring how an individual affects group score is measuring the relevant individual ability directly, rather than by proxy. Then it is a statistical question to find out what an individual contributes to a group.

phyllo wrote:You seem to think that if you get enough data, you will be able to work backwards and calculate how individual contributions produce a group result. The relationships are almost certainly too complex to do that. You're just going to end up with lots of data.

If it's a given that we've gotten "enough" data, then of course we can calculate individual contribution to a group with high confidence (unless we're assuming a degree of agnosticism about how evaluation relates to ability that would undermine any form of evaluation, including SATs and average individual-project scores). Even if the relationship is very complex, with enough data points we can find a model that fits. Each data point increases the confidence.

You are right that I have no idea what "enough" data is, though. But I think, if we take sample data generated by a simple model, and see how reliably we can reverse-engineer the model, we can get an idea. I think we agree that there is some degree of complexity of the actual relationship between individual contribution and group outcome such that it would be wildly impractical to attempt to get enough data for a reasonable approximation. My contention, and where we apparently disagree, is that there are some possible relationships between individual contribution and group outcome that we could get enough data with relatively few data points (for example, if the relationship is the one I used in the program to generate the sample data).

And, I think the data required to establish the model and the data required to actually evaluate students should be treated differently. It could be the case that we need thousands of results to get a good idea of how individual contribution relates to group outcome, but that once we have that relationship, we can plug in a much smaller data set to get reliable scores for a group of students.

Moreno wrote:And really it is not so hard to dip into even a rather big pile of evaluations of a person and learn much more about them in not much more time.

I worked in an academic admissions office that got close to 40,000 applications a year for a few hundred spots, and I can tell you that at that point, the admissions process will use any number they have to pair down the pile. We were concerned to do as holistic a review as we could, but with that many applications it's just not possible, and GPA and test scores would effectively serve as a blunt filter to reduce the applicant pool by more than half.

And, it is my impression that the connection between admissions criteria and actual outcomes is intuition driven and not at all empirical. I would guess that companies that you mention who have moved to group- and problem-solving interviews are actually using empirical findings to improve their hiring processes, since the companies that I am aware of that do this are companies that are willing to buck the status quo to improve outcomes.

Unfortunately, there's a lot of interest in not discovering whether or not e.g. the SAT correlates with future success, not to mention that much of the data necessary to really drill down into what predicts life outcomes is either privately owned or protected by regulations that prevent such analysis. It does seem like that's an area begging for more research (though I am only passively watching for such research; there is likely more out there than I'm aware of).
User Control Panel > Board preference > Edit display options > Display signatures: No.
Carleas
Magister Ludi
 
Posts: 5440
Joined: Wed Feb 02, 2005 8:10 pm
Location: Washington DC, USA

Previous

Return to Science, Technology, and Math



Who is online

Users browsing this forum: No registered users