Benchmarking AMD APU Steam Box

A while back I wrote a blog post about building a Steam Box out of an AMD APU. The idea was to build a SFF PC built around digital delivery of Steam content. The unit would focus on Steam's Big Picture mode, and use a wireless mouse/keyboard and wireless Xbox 360 controllers for input. There were a few things that I needed to work out before embarking on the project. One, I needed to find the case that fit the limited space of my entertainment center. And two, I needed to determine whether a Trinity APU was up to the task of powering a "Steam Box" gaming experience.

That initial blog post came out in June, and a bit has changed since then. IRL has been quite crazy, eating up time I had for component selection and benchmarking. Contributing to this craziness was buying a new house, which ate up most of that time. One good point is that the new house has a great built-in entertainment center, which allows for a much larger case. This article is the fulfillment of the second issue - benchmarking Trinity and Richland APUs.

Specifications & Testing

From a hardware perspective, I tested three APUs: A10-5800K, A10-6700 and A10-6800K. They were all tested on an MSI FM2-A85XA-G65 motherboard, with 8GB of RAM (2x4GB) in dual channel configuration. The memory speed was set manually at either 1866MHz or 2133MHz, as required. All tests were run at stock speeds.

As, I noted in my original blog post, my goal was to make the APU Steam Box so that my kids could play appropriate games from my Steam library. I didn't want to end up repurchasing games for our Xbox or Wii, which I had already purchased on my PC. So, from a software perspective, my focus was on games that my kids would play. To this end I tested: LEGO Batman 2, DiRT Showdown, Rayman Origins, Sonic 4 Episode I, and Sonic Generations. For titles that I might play, I tested: Guild Wars 2, Rocksmith and Torchlight II.

I tested each game at 1080p, at the maximum settings for each game. For some games I tested additional quality settings. The goal of the Steam Box was to replace a current gen console - PS3 and Xbox 360 at the time of original conception.

These consoles rarely render games at 1080p, but instead often render at 480p or 720p, and upscale. And, these consoles often render at an image quality considered "Medium" by PC users. But, we are PC gamers and we expect more. Additionally, this would give a better comparison to the upcoming Xbox One and PS4, which are expected to render closer to 1080p with higher fidelity.

Looking at the specs and components used, the system will run about $600. This assumed you don't have an extra mouse+keyboard combo or controller lying around, and don't have any additional components such as spare RAM or a spare HDD to use. It also uses some premium parts such as PC2400 RAM and a 240GB SSD - which I got on sale at Amazon for $140.

My build components at $600 are decidedly better hardware than comes in any console, and prices closer to the upcoming PS4 and Xbox One. However, the Steam Box doesn't have annual subscription fees from Day One, and no console has SSD storage.

You could also shave off about $200 off this price picking a cheaper motherboard (I had the MSI on hand), choosing PC2133 RAM and a 500GB HDD. This would put you around $400 for everything. Shopping Black Friday Deals and/or dealing with rebates, and you're looking at $350.

Benchmarking

Below is a table that outlines the summary view of the test. It shows the Min/Avg/Max fps scores of each game across each APU. After this summary is detailed information on each game. The game analysis provides a second-by-second fps graph of each game, as well as qualitative feedback on each experience.

DiRT: Showdown

For Dirt Showdown I tested the in-game benchmarking utility. It is a single pass through the figure eight style track. I tested at Ultra and High (mislabeled as Very High on the graph, sorry). Ultra maxes out all the settings, including the advanced Global Lighting options. High drops some settings a bit, and turns off the advanced lighting. A Medium setting should keep you above 30fps at 1080p. High produces very consistent frame-to-frame performance, and each chip shows a steady increase over the previous.

The Ultra settings are completely unplayable. The Very High settings might be barely playable, with an average just above 20fps and a max of 30fps. Dropping to Medium should keep you above 30fps.

Guild Wars 2

The original Guild Wars offered a quality slider for specific steps between performance-minded and quality-minded settings. For any PC configuration, the slider gave the same settings at each step. Guild Wars 2 goes about quality settings in a different manner. There are three options - Best, Auto and Performance. Best maxes out all settings for maximum quality (and lowest performance). Performance minimizes visual fidelity to maximize framerates.

For Auto, the game attempts to detect your hardware and provide game settings that balance visual quality and performance. It appears that the game attempts to set performance at approximately 30fps. In testing, I found that Guild Wars 2 did pick higher settings for the A10-6700 and A10-6800K, versus the A10-5800K. I went back through and retested the A10-5800K at the quality settings produced by the A10-6800K. Below are screenshots of the different quality options. They are in order: Best, Auto, Performance. Click on them to cycle them through the pop-up image "lightbox".

For my test, I did a consisent run outside the village of Shaemoor. I started on the ramp to the south of the Shaemoor Fields windmill. I ran around the windmill, past the Fields Waypoint, and across the bridge. From there I ran a u-turn across the small footbridge which crossed the river, turned north and ran to the steps of the millhouse.

At the Best visual quality, the A10-6800K can only manage an average of 15fps. At the Auto settings, it comes in at an average of 27fps and a max of 33fps. At the Performance level - which sacrifices significant visual fidelity, you come in at just under 60fps on average, with a max of 76fps. Overall the A10-6800K would work well for Guild Wars 2. A little manual tweaking would find an fps that stays above 30fps. All three of the chips provided good performance, with small gains with each step. There is a little chug on the "Auto" graph with A10-6700K. This was an anomaly that I didn't catch in the graph until I had already swapped the chips out. By and large you will see similar performance curves across each chip.

LEGO Batman 2

LEGO Batman 2 is the perfect title for this use case. It's a kid-friendly title, and works great on a big screen with a controller. I tested the cinema level that results in a confrontation with Two Face. There are lots of enemies, and lots of shiny and reflective objects. This is live game play, so there are some differences between each run through. Even with this variability, the results are very comparable.

The scene was tested at max settings, with all visual effects turned on. Even the A10-5800K hits an average right at 30fps, and the A10-6800K get closer to hitting 30fps as a minimum. With just some minimal adjustments, you could keep the minimums above 30fps. But at this current state, the game is completely playable.

Rayman Origins

Another kid friendly, big screen and controller friendly title. I didn't produce any graphs for Rayman Origins, as they would have been very boring. The game has an innate 60fps lock. I played the first level at max settings, and it held a constant 60fps through the entire thing.

Rocksmith

Rocksmith is another great title for testing. It works great on a big screen, and requires smooth consistent gameplay for proper timing of notes on the path. I tested on max settings, and tested the intro bass guitar technique lesson.

The A10-5800K hits an average above 30fps, with a minimum of 29fps. The A10-6700 and A10-6800K push the minimums past 30fps and the faster chip closes in on 40fps average. Looking at the chart you can see noticeable improvement in each chip, and you can see the consistent steady frames per second.

Sonic 4, Episode 1

What's not to love about Sonic? As with Batman and Rayman, it's the target game type I'm looking for here. As with Rayman Origins, I didn't produce any graphs for Rayman Origins. The game has an innate 60fps lock. I played the first level at max settings, and it held a constant 60fps through the entire thing. I was worried as the title required Java to run, but it ran like a dream.

Sonic Generations

What's not to love about Sonic? Sonic Generations. This title runs like a dog, and there is no reason for it. It is slow as molasses in snow. On high settings, the A10-6800K averages 13fps, and only manages 20fps on low settings. For a game built on speed, this is no where near playable. It was an overall frustrating experience. The kicker is that there's no reason for the poor performance. Yes it's 3D, but it doesn't look any better than Rayman - in fact Rayman is a much prettier title. This is just a dog of a console port. Considering the performance of the APUs in other titles, I can't blame the chip on the performance here.

Torchlight II

I loved the first Torchlight, and used it to test earlier APUs from AMD. I haven't had a chance to really dig into Torchlight II, but it's a game I can't wait to set aside time for. I tested the first intro level on max settings. Like LEGO Batman 2, this is realtime game play that has variances from one run to the next. Even with this, you can see consistently higher performance between chips. The A10-6800K tops 30fps, and each chip provided a smooth experience. Given the previous Torchlight title, I know that the volume of enemies and effects on screen at one time will ramp up. You will probably need to tweak some settings downward to have a playable experience at higher levels.

Conclusions

All in all I feel that the Richland APUs will provide a good user experience for a Steam Box, connected to an HDTV used in Big Picture mode. From a couch seating distance turning down some visual fidelity options won't be as noticeable, allowing for mid-range settings or possibly 720p resolution to be used as needed. In many instances 1080p with mid to high settings will be perfectly playable on an APU powered Steam Box. Again, for my use case I'm not looking to play modern shooters on this rig. I'm looking to build a family oriented box, to play games geared towards a family with younger children. Even then poor ports, such as Sonic Generations, will not be playable on this hardware.

Had I written this article last June, i would have given a "Recommended" to the Richland chips (A10-6700 and A10-6800K) as a foundation for a Steam Box. The limitations of that recommendation would be the somewhat limited performance, and the end of the FM2 socket life cycle. Further upgrades to the upcoming Kaveri chips would have required an upgrade to an FM2+ motherboard. I hate knowing that I'm at the dead end of a roadmap.

FM2+ motherboards are now on the market, and are backwards compatible with the FM2 chips. In this light, a current build would offer an upgrade path to new chips and architectures and the recommendation would come with less limitations. With proper expectations, I still believe that the Richland chips will fit the use case I have in mind. And, I'm hopeful that the upcoming Kaveri chips will perform that much better. I still score the RIchland chips as Recommended, but now with less caveats.