Jeopardy! The IBM Challenge

[BigYoSpeck]

Did any one watch this?

I think it's incredible that in a matter of years these guys have created something that took millions of years to come about through biological evolution. It's not perfect obviously but incredibly powerful and something that can get exponentially better as computer power increases.

Anyone who has young children or is planning on children in the future get them studying in a field like this because it is going to render most other occupations redundant. And as soon as the computers can start developing them self...

The prospect of analytical computers is a scary one. Einstein always used to say he wasn't smarter than other people, he just stuck with a problem longer. A computer that doesn't sleep, doesn't get fed up or bored with a problem even if it lacks the quick human intuition has frightening thinking capabilities.

 

[Tornado]

I caught the second episode, and I have to say it seemed rather rigged to me. Impressive, but not as impressive as it made out.

 

[BigYoSpeck]

I don't understand how it could be rigged? You have the two best human contestants of all time and a computer, all being given the same questions and the the computer won.

Where is there even the potential to rig it?

 

[yoman297]

Sounded rigged to me... I saw it too.

Toronto is an American city, isn't it?

 

[BigYoSpeck]

Guys I'd be more than happy to respectfully debate your opinion it's rigged, but you have to elaborate more than that. How exactly do you think they would rig it? I mean you have two human champions who were beaten, and not by a small margin.

The only possible way it could be rigged would be IBM knowing all the questions before hand.

 

[TB]

The only possible way it could be rigged would be IBM knowing all the questions before hand.

It has happened before.

I would hope that Sony and IBM would be on the level, but there were some things that didn't feel right. For example Watson's reaction time seemed to be pretty damn fast in comparison to Ken and Brad.

Regardless, provided it was all legit, it was fairly impressive to watch.

And what was the deal with his wagers?

 

[BigYoSpeck]

It has happened before.

I would hope that Sony and IBM would be on the level, but there were some things that didn't feel right. For example Watson's reaction time seemed to be pretty damn fast in comparison to Ken and Brad.

Regardless, provided it was all legit, it was fairly impressive to watch.

I'm not 100% familiar with Jeopardy but do they have to wait for the question to be fully read before they can buzz? If so and Watson has already calculated it's answer by that point it will always win the buzz in. They quite deliberately made it still mechanically buzz so it was on a level playing field with the humans but being electronic it would always be able to react quicker.

And what was the deal with his wagers?

I would have to presume they were calculated based on it's current winnings, the opponents current winnings, potential winnings left on the board and statistic calculations of how much of those winnings it estimated it could win based on previous performance to arrive at an amount most likely to ensure victory.

 

[TB]

I'm not 100% familiar with Jeopardy but do they have to wait for the question to be fully read before they can buzz? If so and Watson has already calculated it's answer by that point it will always win the buzz in. They quite deliberately made it still mechanically buzz so it was on a level playing field with the humans but being electronic it would always be able to react quicker.

Yes, the answer has to be read in full before you ring in otherwise you're locked out, I'd imagine for a few seconds.

A servo/piston can be set to react much, much faster than a thumb. Watch an episode of Mythbusters where they calculate how fast a normal person can react, how hard they can punch, etc. and then, invariably, dial it up to a "superhuman" setting.

 

[Crash]

I watched the first night and the last night. Very impressive indeed.

The biggest thing though is that like others have said, Watson's reaction time seemed way too quick. Sure he still using the mechanical buzzer, but Watson has the advantage of being able to "press" quicker than a human can, and it can calculate exactly when to buzz in. Remember, Trebek said that Watson can't hear; Watson relies on the text being fed to him. Since the amount of time it takes Trebek to read the question is variable, I imagine that a signal is sent to Watson at the same time as when Trebek finishes the question, signaling to Watson that the question is done. Sure, it can't anticipate the end of the question, but it sure as hell can process and react to that signal pretty damn fast.

It's kind of funny though, there were definitely times in the game when Ken Jennings was visible frustrated by Watson's buzzing speed.

And what was the deal with his wagers?

I imagine that his wagers were based off of some sort of decision and Bayesian analysis, probably using some combination of the probability that Watson would get the question right, the percentage of correct answers that the other two contestants had, the number of clues remaining in the game, the value of the question, how much money Watson had, and how much money each of the other contestants have, plus other stuff.

When watching the show, my roommate remarked how amazing that a computer can do what it did on Jeopardy. I totally agree with him, but on the flip side, it took IBM a room full of computers to achieve this. We humans have all that "computing" ability in just our skulls.

 

[BigYoSpeck]

When watching the show, my roommate remarked how amazing that a computer can do what it did on Jeopardy. I totally agree with him, but on the flip side, it took IBM a room full of computers to achieve this. We humans have all that "computing" ability in just our skulls.

It used to take a room full of computers to do calculations that your phone can now do without breaking a sweat

Can you imagine walking round with this kind of computing power in your pocket in 30 years or so.

The thing I found most impressive though wasn't the computing power, like you said it took a room full of computers. It's the programming to make something that works with 1 & 0 be able to make sense of spoken language.

 

[Robin]

Its not been as exciting as the IBM Deep Blue vs Kasparov chess match in 1997.



http://en.wikipedia.org/wiki/Deep_Blue_(chess_computer)

Robin.

 

[BigYoSpeck]

Its not been as exciting as the IBM Deep Blue vs Kasparov chess match in 1997.

Well that depends on how interesting you find chess

 

[Tornado]

I don't understand how it could be rigged? You have the two best human contestants of all time and a computer, all being given the same questions and the the computer won.

Where is there even the potential to rig it?

TB and Crash852 already brought it up, but the entire thing came down to reaction times.
95% of the questions that I saw in the episode that I watched had either both of the contestants (somewhat more frequently and obviously, Jennings) visibly attempt to buzz in with the answer, but the computer beat them to it. I only watched 20 minutes of the second episode and I knew the thing was going to win the whole event because of that alone.

Basically, impressive or not (and I admit that it was impressive), the whole thing couldn't have been more transparent or telegraphed than it was. They might as well have had the thing playing by itself with the way they had it set up, and my only real surprise was that they were that overt about it.

 

[BigYoSpeck]

TB and Crash852 already brought it up, but the entire thing came down to reaction times.
95% of the questions that I saw in the episode that I watched had either both of the contestants (somewhat more frequently and obviously, Jennings) visibly attempt to buzz in with the answer, but the computer beat them to it. I only watched 20 minutes of the second episode and I knew the thing was going to win the whole event because of that alone.

Basically, impressive or not (and I admit that it was impressive), the whole thing couldn't have been more transparent or telegraphed than it was. They might as well have had the thing playing by itself with the way they had it set up, and my only real surprise was that they were that overt about it.

That's an inherent advantage of a machine though, and as it was machine vs human any kind of artificial handicap would have made the whole exercise useless. That it could come to a confident answer in the time it takes for the question to be presented to a person is the impressive part.

I guess the next step will be not the presenting of a txt question to it but the same sensory inputs as the human contestants i.e. voice recognition and OCR of the screen and to make it dynamically respond to the people it's competing with.

 

[Tornado]

That's an inherent advantage of a machine though, and as it was machine vs human any kind of artificial handicap would have made the whole exercise useless.

The entire point of the thing was to show advances in emulation of human intellect by machines. Not ability to respond to stimulants faster than humans. Anyone who has driven a car made in the past decade already knows that computers can be made to respond to things faster than humans.

Not making concessions to balance out blatant physical advantages held by the machine already made the whole exercise useless.

 

[BigYoSpeck]

The entire point of the thing was to show advances in emulation of human intellect by machines.

I would say that the motivations of IBM were more to show computers superior ability to do tasks currently dominated by human thinking. They key use for this in the long term will be taking current human based analysis and doing it quicker by computer.

 

[TheToad263]

I saw this and I was pretty impressed at what that computer could do. I would like to see some more info on exactly how the system works, but overall i would say nice job IBM.... Deep Blue beat Kasperov and now Watson beat Jennings.

 

[mofo69]

I would say that the motivations of IBM were more to show computers superior ability to do tasks currently dominated by human thinking. They key use for this in the long term will be taking current human based analysis and doing it quicker by computer.

I think it was really cool and all - big supporter for IBM and the computers - but Jennings was trying to ring in EVERY time on the second day, and he just couldn't beat him to the punch. Not exactly fair, but IBM paid TONS for the show so eh.

 

[BigYoSpeck]

I think it was really cool and all - big supporter for IBM and the computers - but Jennings was trying to ring in EVERY time on the second day, and he just couldn't beat him to the punch. Not exactly fair, but IBM paid TONS for the show so eh.

He was trying to ring in every time, likely having not yet arrived at the correct question and hoping to have got there by the time he had to voice his question. Watson on the other hand was buzzing in having already confidently arrived at the question so while it didn't have the human advantage of game tactics, it had the computer advantage of speed which is why it won and in what I consider a fair contest. A machine has the advantage of speed and reactions so it completely makes sense to include them in a contest against a human.

 

[Tornado]

He was trying to ring in every time, likely having not yet arrived at the correct question and hoping to have got there by the time he had to voice his question.

This is an irrelevant qualifier by very nature of the rules of the game, even if you did have the slightest shred of evidence to support it in the first place.

Watson on the other hand was buzzing in having already confidently arrived at the question

Ditto to the above. It doesn't matter how quickly you answer the question so long as you do it in the allotted time. Just like it doesn't matter if you know every answer on the board if you can't ring in because one of the contestants has a dramatically unfair advantage for buzzing in.

it had the computer advantage of speed which is why it won and in what I consider a fair contest.

But it wasn't, and it really isn't open to debate whether it was or not.

A machine has the advantage of speed and reactions so it completely makes sense to include them in a contest against a human.

Again, the point of this stunt wasn't to show that machines could respond to stimulae faster than humans, and not attempting to plan for that made the entire thing meaningless.