Scouting vs. Sabermetrics:  Questions...

Since I have no banner to wave to defend either over the other. There isn’t a barrier at which one stops reliance on statistical analysis, nor is there a gate that locks one into observing the physiology and skill attributes of the athlete. One lets you assess physical capacity, if you know enough about applicable skills. The other is a condensation of the players’ actual performance, and if you know enough applicable analyses, the capability to assess a players’ value with respect to all players, or replacement level (arbitrary but good hurdle) players, peers at his position, or even how best to utilize the player on the field or in a lineup. Talent scout in one case, assessor in the other.

IOW it takes both to make decisions about a player. Decisions include draft, assign, promote/demote, compensate, keep or trade or release, and if experienced, acquire or not. Most of the arguing I’ve heard and read centers over perception of player goodness, or at times, the validity of scouting versus sabrmetrics. Here’s one way I’d express it:

You scout to find ’em. You statistically analyze to apply ’em.

(1) Some of the traditional measures are convoluted and do not directly measure a player’s performance.
(2) It is possible to quantify the role of luck, which has been ignored completely by traditional measures.

The front office revolution isn’t about using one or the other but in learning how to integrate information from both sources. It might be about one or the other for some journalists.

Keep in mind that Sabermetrics were basically started by a fan wanting to more accurately evaluate players. That’s not to say that teams weren’t creating similar metrics but the first published stuff was created by fans for fans. To me, I see the metrics/stats as devices for fans to use in various arguments. I may see an ERA but someone else can discount that with peripherals; I see a high OBP and someone else notices an unsustainable BABIP.

Here’s the seminal piece on scouts. Is it truly relevant to this discussion? Maybe. Is it really funny? Probably.

http://www.firejoemorgan.com/2006/08/best-ever.html

About ten years ago, I was out looking to buy a used sportbike. Yamaha FZR 600 to be exact. When I test drove it, I notice that the transmission was exceptionally klunky. Sounded like lots of metal contacting metal. Completely turned me off from buying it. Later, when I got on the internet, I noticed that several reports on this bike had mentioned the klunky transmission. They all gave similar reports. Yes, it was klunky, but it was not something that ever needed repair. So I went back later and bought the bike. The bike never had a mechanical failure while I owned it, even though I had a minor wreck on one occasion. The point is this some things you have to see for yourself. Other things are best explained through long term statistical analysis.

In baseball, things like speed, hustle, leadership, toughness, handling minor injuries, emotional demeanor, and making adjustments are all better seen in person. (Scouting)
Other things like batting average, power, plate discipline, walk rate, and strikeout rate are all better viewed through stats. My personal bias tends toward the latter for 2 reasons. I think they have more of an impact on winning and losing ballgames, and I like playing with numbers. One curious side note: I don’t really know if defense fits more on the scouting or stat side of the equation.

I am a very quantitative person. I love stats, and think baseball, more than any other sport, is beautiful for its ability to be captured by numbers (yes, I’m a nerd for saying that). That said, I think there is a tendency of certain people to over interpret stats. Its human nature, after all – you discover some knowledge that others don’t have, then it is in your interest for that knowledge to be applied to as many aspects of your field as possible.

For instance, I love the metrics designed by folks to quantify wins added by players – from the classic Bill James win shares, to WPA and RAA. Seeing those numbers on fangraphs is fun and revealing about which players contribute the most. But at the same time, the tendency for those same people to try to estimate players’ values in the future frustrates me. Projection is a far, far different science than description. And I don’t think the “Marcels” projections (as well as those projections by Bill James and others) have much value at all in predicting an individual’s value. The dollar value projections, which I gather were generated by Dave Cameron, are borderline insane to me.

So that is my long-winded way of saying that I think stats are the ideal (if not only) way of describing how a player was. But I’m not so sure that people have devised accurate methods to determine what a player will become. Until then, I think scouts, intuition, and common sense are often better predictors.

passed down from baseball immortals. It really wasn’t about a particular player’s performance. It was more about strategy. The famous example being that of determining whether a slugger should hit in the fourth spot just because of the ability to slug the ball without any consideration of an ability to get on base. As we all know, he found OPS was far more important that SLG (or HR) alone.

Player evaluation is a natural evolution of Bill James’ early work. For example, if the more effective power hitters are judged by OPS rather than SLG (or HR), then when a team need to add a power hitter that is the statistic they should look at closely to evaluate those that are available.

In projecting players, the sabermetric approach, as I see it, is complementary to scouting. Both attempt to identify attributes of successful major leaguers that they can see in budding form in younger players. Scouts look for physical characteristics like bat speed, swing mechanics, arm speed, action of pitches, and so on. Sabermetrics measure outcomes, and may measure physical characteristics in the future.

So, a minor league player that gets great outcomes with limited physical characteristics is still a long shot to make it in the majors. A player with great physical characteristics but horrible outcomes might have a better chance to make it in the majors but may have a very limited role. In fact, the scout opinions could be used within the sabermetric approach by assigning index values to physical characteristics deemed important to scouts. Or, you could just use the 20-80 rating system and stratify prospects by both the scout rating system and some collection of saber stats.

I don’t go too far beyond outcomes when using sabermetrics for player projection. It’s important to examine the outcomes relative to league performance and any odd ballpark effects, but aside from that I don’t think there is much to gain by more advanced techniques for adjusting outcomes to rank the players, like the runs below replacement metric that is used to rank major leaguers.

Examples.
Strategy
In the traditional use of sabermetrics, the Rangers can look at the numbers and easily see the ballpark favors lefties and, therefore, target LHSP and LH power hitters to anchor their rotation and lineup. They could go an extra step and examine scenarios, like would the backend of the rotation be better with three league-average lefties or three league-average righties. That is, would the performance of the league-average lefties be slightly better than that because of the park characteristics (and righties slightly less) to the extent that the difference isn’t due to randomness.

Prospect Evaluation
I live no where near Arlington or the Rangers minor league affiliates. I rarely get to see players in person. When I do, the best I can do is compare them to how they look relative to others on the field. Some guys I have seen are obviously better than the other folks on the field. In Spokane, Davis had more power than anyone I had seen that year or in year’s previous. His mis-hits and one-armed hits were doubles. Littleton was better than anyone at keeping the ball low. Boggs played very shallow in center field. At Clinton, Main threw more fastballs low in the zone and got more swing-and-misses on fastballs than anyone else. Cristian Santana’s bat speed was way beyond everyone else.

What’s the point? I can’t possibly say what my small set of observations may mean about the quality of the Rangers’ prospects. I have to rely on Jamey, MJH, Z, JP, Cole, Sickels, BA and others. Or, I could just stop being an analytical, obsessed baseball fan. Right.

Maximiliano Ramirez
About 20% of plate appearances end up as a K, about 17% as a BB, about 31% as a H. His ISO jumped from .200 to .300 in the past year. All rank well compared to his peers in the Texas and California leagues. The scouts say his best attribute is that he has a mature approach. That is, he knows what he wants to swing at, and he doesn’t swing at bad pitches. I have never heard a scouting report that says he has exceptional bat speed.

My view of Max is that he could hit .300/.400/.600 in the major leagues. I wouldn’t have said that a year ago. Even if he doesn’t have superior bat speed compared to other major leaguers, the scout’s opinion of his approach might mean he could reach his peak level even without superior bat speed. He’s just really, really good at recognizing what the pitch is, where it is going to go, and whether he can hit that pitch. I really believe that his bat alone could make him an All-Star.

If he didn’t have one of either superior bat speed or a great approach at the plate, I would say there is no way he could reproduce this year’s .300/.400/.600 AA line in the majors.

Perhaps in the future physical characteristics like swing arc, bat speed, pitch movement/location/speed will be measured and incorporated into a saber approach. Until then, and maybe even then, scouts are really the only thing that can tell someone HOW the outcomes are produced.

There are some indications that pitch f/x can do some of what scouts can do. For example, showing that high elevation at Coors has less of an impact on changeups than curveballs ad sliders. But, pitch f/x is far from being a replacement for human eyes at this point.