Author Archive

Building a Retrosheet Database – Part 1

That magical time of year has come again. Yes, the very fine folks over at Retrosheet have updated their game files once again, and the new batch includes (among other updates) the play-by-play files for every baseball game in 2014. There are a lot of things you can do with the Retrosheet files, but one of the most powerful options is to create a Retrosheet MySQL database.

Certainly, there are a lot of things you can find out at FanGraphs and Baseball Reference without the need for your own database. But every now and then, a question pops up that Googling, FanGraphs leaderboards, or even the Almighty Play Index at Baseball Reference cannot answer. It’s a luxury to have your own database, to be sure, but any baseball nerd worth their salt should at least have one to tinker with.

This tutorial is going to be starting from scratch. This means that this first installment of the series will simply be about acquiring the right tools to build out the database. We’ll get to actually inputting the baseball data in the next chapter. This may seem like the boring part, because it is. But it’s important to have the proper foundation before we get to the fun stuff.

DISCLAIMER: For right now, I’m going to be working strictly in a Windows environment. The tools that we need — which will be revealed later — were meant to work with the Windows command line. There are options for you Mac/Linux users out there, and I will discuss them down the road. But for right now, these initial steps will be for the Windows folks.

MORE DISCLAIMER: I’m trying my hardest to keep this tutorial accessible for everyone. You’ll need a basic understanding of Windows, but you shouldn’t have to be a computer wiz to follow these steps. Conversely, those that are very familiar with Windows will see this as pretty basic stuff. I’m not trying to be insulting. I’m just trying to make sure everyone can follow along.

Step 1: Installing MySQL

The term “database” might sound like you need some sort of special hardware or server rack in your house. While this is true in the corporate world, the truth is you can have your own database right on your regular PC. No need for special hardware or hosting plans or anything of the sort. This is not to say that having a hosted database doesn’t have it’s advantages, but for the time being, we’ll be dealing with what are called local databases — those that live right on your regular machine.

However, a complete Retrosheet database will require a good deal of hard drive space. From my experience, a Retrosheet database that spans all the available data requires a little over 8 GB of hard drive space. However, as Retrosheet releases more and more updates, that need will expand. To be safe, I wouldn’t work with any less than 15 GB of available space. You’ll be surprised how fast that fills up.

Personally, I have a separate machine that hosts my personal database. It’s a headless (just  a tower stowed away in my attic, no monitor or keyboard attached) unit that performs a lot of functions like storing my ripped DVDs and music. But my Retrosheet info also lives there, and I connect to it to do my querying. If you are totally baffled by this notion, don’t worry. I’m still teaching you how to install it locally. But for those with a little knowledge of how to handle some basic home networking, know that this is an option.

OK. Let’s get to the action. The type of database that we’ll be using is called MySQL. It’s a very common database language. There are lots of resources in both book and online form to learn more about it, and pretty much any problem is easily Googlable. It’s one of the industry standards, so that’s what we’ll deal with.

To keep things nice and free, and since we’ll be only using this for personal reasons, we’ll deal with the Community Edition of MySQL. You can find the download link here. You’ll want to choose the second (larger) download link toward the middle of the page. The next page will give you some buttons to login or signup, but you can bypass that with the “No thanks…” link a little further down the page. Once the .msi file has downloaded, go ahead and run it.

pt1screengrab1If you want to go the quick and dirty way, go ahead and choose the Developer Default option. It will install everything you need, and probably some stuff you don’t. You might want that stuff later on, but for the sake of this article, I’m going to choose the Custom option and pick what I want installed. You should install the Server and Workbench at minimum, though it’s a good idea to include the notifier and the documentation as well. You’ll have to click through some category trees to get what you want. Use the green arrow to add your options to the list of things to be installed.

pt1screengrab3The installer will check to make sure you have all the required utilities to do what you want. In my case, I didn’t have the C++ 2013 distribution on my machine — you may have more things that need installing. Just click Execute and the necessary files will be downloaded and the respective installers will run. Once everything is installed, click Next and your MySQL install will begin.

pt1screengrab5For configuration purposes, go ahead and keep the defaults, unless you have specific reasons not to (and you know what you’re doing).

pt1screengrab6You can add users to the database if you want, but for simplicity, we’re just going to define a password for the “root” user. REMEMBER THIS PASSWORD. It’s your key to doing anything with the database — adding data, querying, etc. If you lose it, you’re pretty much horked. Use a familiar password or write it down. If you’re just having this on your own machine, there’s really no need for a super-complicated password. It’s holding freely-available data, after all.

pt1screengrab7For the Windows Service configuration, you can keep the defaults as well. You can choose to not have the MySQL service to run when you start the machine, but if you do you’ll have to manually start the service each time you want to use the database. If you don’t know what that means, have it run on default.

pt1screengrab8Go ahead and click Execute, and the options will be configured. After that, a few clicks of Next should do it. You’ve just installed MySQL on your machine. Congratulations!

Step 2: Install Wget

Wget is a great utility for downloading mass amounts of data from a server without having to click a thousand links. It’s a script-based tool, and it’s what you’ll need to download all the Retrosheet data with in one big swoop. Don’t worry, we’ll provide the scripts you’ll need.

Download the installer here. Run the installer, keeping all the defaults.

Step 3: Install 7-Zip

7-Zip is a .zip file utility. It’s one of the best free tools for handling .zip files, and it’s what we’ll be using. You can download the installers here. Choose one of the first two options, taking care to choose the right version in regards to 32- or 64-bit. If you don’t know what version you have, go to the Control Panel of your machine, click System and Security, then System. You’ll get a similar screen, which should show you your version. Remember which one you have, as this will be important later on.

pt1screengrab9
Yes, I name all of my devices after baseball players. Don’t judge.

After you’ve downloaded the right version, go ahead and install, using all the defaults.

You’ve done it! You’ve done all the pre-steps for installing your Retrosheet database. If you have any questions, sound off in the comments. Stay tuned, as next week, we’ll get our hands dirty with installing the actual Retrosheet data. I’ve made it as painless as possible, I promise. Don’t get worried. If you can handle what we did today, you can handle the rest. Good luck, and don’t feel bad about asking for help.


Making GIFs Might Become Easier Thanks to YouTube

Though the technology is a little outdated, GIFs are still the de facto method for sports fans and writers to share quick videos for entertainment and analysis. There is no shortage of methods to create GIFs, but it’s often a time-intensive process. Usually a minimum of two programs are needed to capture and convert the MP4 or FLV files. But now, online video giant YouTube is slowly pushing out a tool to help ease that process.

The tool is only available on a very select collection of videos at the moment. Clips on the PBS Idea Channel are some of the few to get the feature as of this writing. The tool is baked in right into the sharing function already found in YouTube. Right next to the direct link and embed options is one labeled GIF. The user can choose up to six (consecutive) seconds to be converted with some simple visual sliders. The option to add text to both the top and bottom of the image is also available.

Image via YouTube
Image via YouTube

If the feature gets adopted on a broader scale, it would obviously be great for those of us who love to bombard our friends with cat clips and Russian dash cam videos, but it could also be very useful for the sports fan. The NBA leads the way in YouTube compatibility with their laxer rules regarding fan postings and their own prolific uploading practices. This new YouTube option would make it much easier for folks to post monster dunks and buzzer beaters to Twitter or their own blog pages. The NFL and MLB are getting better about YouTube postings, though they seem to only tolerate the heavily-curated videos they post themselves. This may be less handy for the budding analysts out there, but it would still be great to be able to send around a snippet of, say, this amazing Bo Jackson clip:

GIFs will most likely be phased out in the coming years in leu of better technology, but the fact that the biggest player in online video is at least testing a way to better create the easily-sharable file is an interesting development.

(Header image via Rego Korosi)

Hockey Teams Are Turning Ice Rinks Into Projection Screens

Hockey is seemingly always a fun event to watch live. Even if your favorite team isn’t performing so well, the speed of the game, the always-present chance of a fight, and the overall live experience of live NHL hockey is an enjoyable one. Fans seem to agree, as — at least in the short term — NHL attendance records are growing. Some teams are not resting on their laurels, however, and are instituting even more ways to engage fans that make it to the arena.

devilsice

That bit of visual mastery can be found at The Prudential Center, the home of the New Jersey Devils. It comes courtesy of a dozen high-powered 1080p projectors that are mounted to the rafters of the arena. The company behind it is Quince Imaging. They have set up similar systems for other NHL teams as well as a few NBA clubs. The company was profiled recently by Wired.

And according to Quince COO Scott Williams, it doesn’t have to stop at fancy pre-game theatrics. There are possible plans to integrate interactive games — even some controlled by fans’ mobile devices.

The detail of the graphics is impressive in both the GIF above, and the video below of the Calgary Flames’ intro. We are only a few years removed from HD scoreboards over center ice, and now arenas are tricking us into thinking people can freeze fire with their skates. I’m sure the addition of similar systems in other arenas would be a welcome change for fans.

(Header image and GIF via Quince Imaging Vimeo page)

MLB 15 The Show Is Bringing You Year-to-Year Saves

It was always the big paradox. Whenever a new rendition of a sports game came out you had two choices: upgrade and get all the new graphics and features, or stay the course with your current version and continue to build on the sports empire you had created. Well, luckily, MLB 15 The Show is making it so you don’t have to make such harrowing choices. Finally, FINALLY, you will be able to carry over your progress from the previous version of the game to the next.

According to both the official trailer and the list of new features over at GamesStop, if you have given your blood and sweat to crafting an unbeatable juggernaut team or your perfect Steve Nebraska-type player on MLB 14, you can take that progress with you if you procure MLB 15. It’s not 100% clear how the Franchise or Road to the Show game saves will be transferred — I suspect an Internet connection will need to be involved — but the mere fact that it’s being mentioned should cause fans of the game great relief.

This feature would be welcome in any sports game, but even more so in a baseball offering. The seasons are so long and the process to get created players to the majors is so exhaustive that it always brought me a tinge of heartache to leave a game behind for a shiny new one. The prospect of trying all over again always brought some excitement, but I found myself wondering whatever happened to the lefty knuckleballer I made or the unstoppable Mets team I curated just a few years back. This new feature will be celebrated a great deal in my living room, certainly.

You can check out the other new features — including some sort of radio show implementation — at the GameStop link above. The game is set to be released (friendly reminder: PlayStation systems only) on March 31st.

 (Image via Youtube)

Bluetooth 4.2 Could Be a Boon to Wearable Tech

There’s a scene in Iron Man 2 where Tony Stark is struggling to create a new element for his chest piece. He ends up watching an old promotional film his father created, which includes an Easter egg at the end specifically for Tony. In that speech, he uses the phrase “I’m limited by the technology of my time.” It’s a plot point, but it’s also a phrase I think of often. So much of what we see in the tech world, and what we cover at TechGraphs, is straddling the bleeding edge — using everything currently available to us to create the best and most innovative products. But there’s a limit. Moore’s law aside, there are only so many advancements that can happen given the current hardware availability. Every so often, there needs to be a jump. And if what the Bluetooth Special Interest Group says holds true, we may be on the brink of one of those jumps.

The Bluetooth SIG recently announced the coming of Bluetooth 4.2. You can read about all the upcoming changes for yourself, but the big enhancement centers around something called low-power IP connectivity. This essentially gives new Bluetooth devices the ability to connect directly to the Internet — no intermediary required. The ramifications of this has underpinnings in everything from automobiles to lightbulbs, but one of the biggest benefactors could be the wearable tech industry.

With direct Internet connectivity, Bluetooth devices will no longer need to paired to some intermediary device (e.g. a smartphone, tablet, or laptop) to upload and download what it needs to function. It will still need a connection to the Internet via some sort of gateway device, so it’s not as if our future smart watches will be totally free, but it still loosens the restrictions on what a device needs to operate.

Many current offerings of helmet sensors, for instance, use Bluetooth connectivity to relay data back to some central hub. Again, this would be a smartphone or laptop. That laptop then takes that data and uploads it to some server somewhere. This all works fine. Unless the laptop crashes. Or there is interference with the Bluetooth signal. Or the devices roam out of the field of connectivity. Most devices plan for this, certainly, by including some sort of on-board storage that would hold the data until the connection is restored. But if the helmets were able (through an in-stadium device) to upload the data right away, without the need for an extra connection, data could end up being more reliable and devices — not needing the extra storage — could get even smaller. There’s also an increased ability for these devices to talk to each other, be they in-equipment sensors on a hockey team or multiple sensors on one athlete.

This is a little bit of speculation, of course. We will have to wait until the protocol is widely adopted before knowing what kind of advancements can be made. And I’m sure there are things that you or I haven’t even thought about that will make their way to market or to the research sector. And that’s kind of the fun part. While it won’t bring about the Stark vision of a clean, endless source of energy, some pretty interesting things can come about when we are no longer limited by the technology of our times.

(Image via Manuel Iglesias)

What Vine’s New Push Notifications Could Mean for Sports

The social media app Vine is sort of hard to nail down. Yes, it’s a tool that lets users share short videos with their friends and the world, but it doesn’t have one particular purpose. People use it to make comedy clips, post super short music videos, commit acts of journalism, and any other kind of thing that can fit in a six-second short. Vine is a great tool to blast out a video to all your Facebook and Twitter cohorts. But with our list of friends and followed accounts growing all the time, it becomes easier and easier to miss great content. Vine is hoping to counteract that by offering users the ability to receive mobile notifications from their favorite Viners. With a single touch, people can now get notified whenever an account posts a new video.

This could be seen as useful for lots of people, but I think sports fans could really benefit if leagues and teams do it right on their end. Almost every major league sports franchise has a Vine account. How much they use it varies quite a bit. But Vine could be used by these teams (and their respective leagues) to push out a good deal of highlights to fans. Certain league-specific apps can do this already, but Vine could be used as a one-stop highlight spot spanning across multiple leagues and teams.

It’s true that some teams use Vine to do this already. But the addition of push notifications taps into the immediacy of sports. Of course people can gather around a computer at the office on Monday and re-watch (and re-watch and re-watch) Odell Beckham Jr.’s one-handed snag, but a big football fan would want the ability to see it on their phone while they’re at the airport, on the bus, or even hanging at home without the game on TV.

Vine’s new push abilities compliment sports highlights as a whole. There could certainly be some TV licensing rules to deal with, but teams and leagues should see this new feature as a great potential to reach fans. We want our highlights and we want them now. Vine is making it easier for that to happen. Now it’s on the content creators.

(Image via Jason Howie)

ESPN Planning Debut of Pay-to-Stream Service with Cricket World Cup

We’ve all heard rumblings of ESPN launching services that allow viewers to watch MLS or NBA games without a cable or satellite subscription. As of today, they’ve only existed in rumor. According to a report by Re/code, however, ESPN may finally be entering the cord-cutter-friendly domain as early as February — though not with the sport you might assume.

Though plans don’t seem to be quite finalized as of yet, ESPN is looking to broadcast the Cricket World Cup via online streaming service come February. To U.S. viewers, this may not hold as much clout as a deal with the NBA, but it would be the first step in the eventual move away from TV-bundled streaming. As of now, services like WatchESPN and Fox Sports Go require a current cable or satellite subscription for full functionality. This makes it a convenience service for current subscribers, not a solution for those who don’t have the need/disposable income for a TV package. What ESPN is supposedly offering with cricket would pave the way to a much wider a-la-carte option — only pay for the sports you want to see.

This isn’t a shot across the bow of the streaming services already offered by the four major American sports. Were this type of streaming service to expand to leagues like the NBA or MLS, it would most likely come as an addendum to a cord-cutters current arsenal to go along with their current MLS Live or NBA League Pass subscription.

In a perfect world, we’d be able to pay one lump some to stream all the [insert sport here] games that are played during a season. But with team revenues tied so strongly to TV contracts, that doesn’t seem very likely. Certain leagues have deals with certain TV networks, and that’s just a fact of life. But if we were allowed to pay that network a little extra to access those usually-blacked-out games, that may be beneficial to both sides. Though it’s getting its start in a fairly obscure market (at least as far as U.S. watchers go), ESPN’s plans with cricket might be that pivotal first building block.

(Image via Pulkit Sinha)

Fix Your Golf Game with these Amazon pre-Black Friday Deals

We’re still a little bit from the trample-fest known as Black Friday but that’s not stopping Amazon from getting on the deal game a little early. They’re running sales at their site all through this week and next, and the enterprising golfer can take advantage if they act soon. Both the GolfSense 3D Swing Analyzer and the SwingTIP Swing Analysis system can be had for a steal today providing you get in before the deals are gone. I suppose “steal” is a relative term, as the sensor+app systems are still fairly pricey even after the discount. But if you’re looking to use tech to improve your game in the offseason, Amazon’s got your back today.

The mega -retailer is promising a slew of deals over the next coming days, so whether you’re in the market for some wearable tech, a new tablet, or even a new laptop, it might be wise to keep checking back. If we come across any super deals for the sports-loving techie in our travels, we’ll be sure to pass them along.


ShotTracker Uses Sensors to Up Your Basketball Game

The idea of using technology and analytics to improve one’s skill set is no longer relegated to the pros. There are already plenty of offerings available to the consumer to help them with their tennis/golf/baseball/etc. And while other companies have tried to breach the basketball market, ShotTracker is touting a new easy and (fairly) affordable way to track your improvements on the court.

The idea behind ShotTracker is fairly simple. A combination of a wearable wrist sensor and a device secured to the net of the basket work together to help track shot attempts, makes, and misses. All that data is then beamed to a smartphone or tablet for further analysis. But the app goes beyond simple counting. Users can create custom drills and workouts, take a look at their shot map, and even follow the progress of  friends and teammates.

ShotTracker's shot map feature. (Courtesy of shottracker.com)
ShotTracker’s shot map feature. (Courtesy of shottracker.com)

ShotTracker also offers functionality for coaches. Custom workouts can be created and assigned to players. Coaches can also see which players have been practicing and which haven’t, and get a breakdown of who is improving in which areas.

ShotTracker provides something that can often go missing in practice sessions: structure. It becomes easy to just kind of mess around without knowing if you’re improving or not. With the ability to create drills around a particular skill or facet, and a way to empirically gauge  improvement, ShotTracker takes the guesswork out of practice.

ShotTracker is available now for $150, which includes a wrist sensor, wrist band, shooting sleeve, and net sensor.

(Header image courtesy of shottracker.com)

WATCH: How Data Scientists are Mapping the NBA

If Kirk Goldsberry’s name doesn’t ring a bell, his work should — especially if you’re an NBA fan. Kirk is the guy behind those super-informative NBA shot maps over at Grantland. Goldsberry is a professor at Harvard, and works on utilizing the new SportVU system to track all kinds of shot data from around the NBA.

Blake Griffin shot chart courtesy of Grantland.com
Blake Griffin shot chart courtesy of Grantland.com

He’s also a subject in a new book by Wired editor Mark McClusky. McClusky is the author of Faster, Higher, Stronger: How Sports Science Is Creating a New Generation of Superathletes–and What We Can Learn from Them. McClusky profiled Goldsberry in a video short for Wired. It’s a nice look behind the scenes of the ever-growing field of  sports analytics and sports data journalism. Check it out below.

(Header image via Grantland)