Blogging from ISCA: ACLD: Chuck Thakker, Rethinking Data Centers

Chuck Thakker invented the PC and Ethernet.

Chuck is now a post-PC person. He has no laptop. He survives with a SmartPhone. (It wasn't clear if he showed a Windows Mobile SmartPhone or an iPhone.)

Datacenters... shipping containers full of servers.

Anecdote: shipping containers were designed as a system... they succeeded where others had failed because he made them into an ISO standard.

Anecdote: SF longshoremen fought containerization of Port of SF. Oakland did not. SF is no longer a commercial port.

Container advantages:

Side to side airflow not impeded by the server case. There is no case. The container is... Loop airflow. Big impeller fans.

Shock mounting at the server, not the rack.

Aggressive ideas:

Once through water: pump through datacenter, and then on to farms.

Power distribution: reduce conversion steps.

12VDC->1VDC point of load regulators ~90%

AC->12VDC 2 stage 85%. Can do better: combined power factor correction, etc.

AC transformers 98%

Final efficiency ~80%

UPC and backup generators aren't part of the picture until the grid fails.

Datacenter close to big hydro dam => reliability. Not cost.

Now 1 phase AC to rack. Direction 3 phase AC to rack. Balance, lower ripple. 12-20VAC? Select to maximize overall efficiency.

Commodity servers: HP, Rackable, others.

Thakker: "The PC ecosystem is awful. Terrible." IBM non-x86 servers not in PC ecosystem. "Design our own..."

Custom motherboards.

Commodity disks

Cables at front

Redesign power supply

Thakker: whats ECC / checking.

Use lower power processors, notebook rather than server. (Note, opposite HP Kauffman.)

Network switches expensve. Unreliable.

Data center network:

fixed top

limited nodes

no broadcast/multicast

security simpler

load balance

design goals:

get rid of large switches

push complexity to edge

standard link technology, but not standard protocols

he likes monsoon.

short copper, long optical

2 kinds of switch: middle of rack, boundary. Both can be implemnted using Xilinx Virtex 5 FPGAs. Prototypable...

rack

boundary: 10 Gb ports, 64 copper to rack, 64 optical outside container

rack 20 1+1 Gb to servers, 2 x10Gb to container boundary switches

AFG Q: what switches are outside the containers?

route controllers. (I am not sure if those are outside the containers)

likes monsoon valiant load balancing.

boundary switches are synchronous. source routed. less packet like. long lived connections.


Objections:

Commodity hardware ... network switches to connect commodity parts NOT commodity.

...

Conclusion:

Treating data centers as systems, and do full system optimization.


Questions:

Thakker likes ATM. ATM better than Ethernet cause it doesn't lose packets. Chuck allowed to say this, since he invented Ethernet. But IP community refused to cooperate with ATM.

I asked him a question about switches outside the containers. He said, none: the datacenter is nothing except server containers and the NOC (Network Operations Center).

I asked about containers of switches. Chuck had thought, but doesn't like reliability.

Audience Q: Server computers have a 3 year lifetime. The container itself - power, cooling - can be reused - send back to manufacturer, and upgrade computers.

Compare to servers with cooling integrated: then the cooling solution lifetime is reduced to the computer electronics lifetime.

Audience Q: free air cooling. Thakker: outside air variations. (AFG: Think, winter/summer in the Dalles.) Running electronics hotter bad, reliability.

Audience Q: is power density going to increase. Thakker: microchannels, etc., bad. 8KW/rack now. Thakker doesn't think we will see 24KW/rack.

Mark Horowitz: no more than 10 cores per chip.

Blogging from ISCA: AMAS-BT: Pardo, Crusoe

Transmeta Crusoe

VLIW 5 wide

Generic simulation support

Shadowed registers, commit/abort

Gated store buffer - 32 entries x 32 bytes.

load-and-protect - like ALAT (hw trap)

x86 condition code support.

PC support - low memory steering I/O vs. DRAM. A20M. Crusoe had hardware for memory map.

Crusoe: x86 ISA support, entirely software decode.
Pardo argues that x86 decode big & power hungry.

LongRun, voltage scaling. E.g. leave CPU at 90%, mem at 100%

Shade: 100 inst/sim inst. Perf 3:1 int, 1:1 FP.

Crusoe translation: 10,000 inst/inst.

Schedulig to VLIW target harder than Shade's RISC SPARC target.

x86 reuse rates low.

Crusoe summary:

Reliability, x86ness - good

Cost: good 1/2 Intel/AMD

Power: good 1/3 Intel/AMD

Perf: umm...

Crusoe faster than low power parts, but slower than 15W laptop parts.

Compute bound, often faster at lower watts. But there aren't that many compute bound workloads.

Memory/cache traffic: slower.

Low reuse: translation overhead -> slower

Crusoe has system gotchas:

PCI graphics, not AGP.

Software DMA, not overlapped.


How to do a small project:

Automate, automamte, automate.

Reference simulator. Must be fast enough go boot OS.

Fast VLIW simulator (for host): 30 inst/sim inst. (30 I/I)

Never published?

Narrowing: Reverse execution. Cosimulation, compae. "Nexus" binary search for first divergence. => Bit 17 in register 5 is wrong in this context ... I.e. nexus = automatic bug narrowing.

Testing:

conventional, hand written

Random. Biased to interesting cases.

"Test" means "checkable". Crash => failure. Consistency check => suspicious.

AFG Q: MP non-determinism.

AFG: one nice aspect of SW DMA would be reproducibility.

Reverse HW execution in HW VLIW debugger started, no finished.

Single step trough nested fault handling. A complete debuggeris totally transparent.

Fast builds. Check in early and often.

On failure: binary search of checkins.

War stories...

CMS SW allowed working around many hardware bugs.

Not all reg resources were shadowed. Some bugs due to rolling back after non-shadowed state changed. Added rules checker to catc future bugs.

What Pardo would do differently:

Hardware was bottleneck. Changing ISA was *different* for software teams.

Big projects rules of thumb do not apply to small teams.

Better perf studies from get go. (Threw out 1st 2 CMS.)

More software inspection.

--

CMS written in C, gcc extension to provide HW access. More control.

Modest amount of assembly. Modest amount. Including modest amount of self modifying VLIW assembly code.

Not MP.

Blogging from ISCA: BIC: MATLAB, CUDA

This just a placeholder. Must take more notes for Pardo.

Blogging from ISCA: AMAS-BT: Pardo, SMC

Dave Keppel, Pardo, Google: Self Modifying Code

Everyone knows SMC is dead, but SMC is alive in the very tools that complain about it: dynamic optimizers, dynamic linkers, etc.

Pardo talks fast.

Detecting SMC via page protection. Slow if data and code in same page.

BitBLT - recomile every 10K instructions.

Debugger watchpoints. Change immediates in code.

Present in real commercial workloads.

Coherency events:

x86 - non.

Hardware instructions: ISCP "something changed". iflush addr. coherency(base,length).
Poor match between application and simulator/emulator. Need to detect what really changed.

Adaptive: default-write protect. Change strategy if too many faults. Fall back to default after a while.

Self checking strategy: check current ibytes against saved copy of original ibytes.

Pardo noted that invalidated code often reappears - thing like debugger watchpoints maychange back to original. "Revalidation". Another use for invaid cache entries.

Shade. SPARC. Iflush addr. But, there were some applications that did not use iflush, but which worked on real hardware.

Transmeta Crusoe. Subpage write protection.

"Fetch imediates" - translate code, but fetch immediates that might have been patched.

Crusoe: lots of retranslations when falling through the gears.

Deoptimized translators: fetch immediates. Translation calls interpreter.

Bad: BT leads to more implementations, more chances of bugs, reduced test coverage.

Performance stability, lack of. Consistent sometimes better than fast.

SMC/ISC. Q: what does ISC stand for?

Hardware support:

Crusoe: 2 write protect bits per page. Subpage WP cache.

Shade: 100 instructions to translate an instruction.

Gill51 - universa simulator.

Blogging from ISCA: AMAS-BT keynote

I'll be at ISCA the next few days.

Today: AMAS-BT workshop.

Antonio G. keynote.

Unfortunately, Antonio G and I share the same initials,AG. I will annotate him by Antion, me by me, or AFG.

Pollack's Rule: I suppose I should be gratified that one of my laws, perf = sqrt(power), is now widespread. I am somewhat chagrined that my old boss, Fred Pollack, has his name asociated with it. He publicized it in some keynotes.

Somehow perf=sqrt(power) has also crept in. And Antonio is multiplying the effects. This was not part of my, or Fred's, formulation. Perf=sqrt(area). We often assume that power, at least leakage, is 1:1 with area, which would imply perf=sqrt(power). But I do not think that this needs to be the case. Leakage may be negligible, and active power also seems to be proportional to sqrt(area). Or even less. This implies that perf=sqrt(active power), or less.

Antonio says that multicore => 1:1 perf increases. 2 cores => 2x parallelism. Q: is this correct? The old rule of thumb is that MP, too, perf=sqrt(#processors).

Antonio makes the EPI = Vdd**2 * Cdyn + Leakage. Handwaves leakage. Says Vdd cannot be lowered. (Me: is this true? Differential signalling?) So argues about Cdyn.

Guest ISA / Host ISA. Me: although part of the story, the real challenge is not BT fro ISA to ISA. The real challenges are (a) coming up with a host uarch and ISA that makes sense - that would make sense if compatibility was not a requirement. (b) Minimizing the cost of dynamic instrumentation and optimization.

I.e. the basic host ISA and uarch must make sense, irrespective of the guest ISA.

With an exception: possibly the host ISA is big, with lots of hint bits. The guest ISA may be small and compact. In this case, perhaps the act of binary translation itself helps. The guest ISA may be considered to be a cpmpact form of the host ISA, with just the semantics. The host ISA may be considered to be an expanded form. The host ISA may be considered to be a cache of performance annotations to the guest ISA.

Antonio: memory checkpointing. AFG comment: easier to BT single threaded or message passing programs, harder shared memory.

Antonio: adapting hardware. Resizing, power gating. AFG: hardware can do simiar adaptation. Software dynamic must have larger time constants.

Antonio: BT advantages include compatibility, both over time and across different microarchitectures (which he calls scalability). He notes that forward compatibility is especially interesting. E.g. old binaries taking advantage of new hardware features, like longer vector registers.

Pardo asked about soft real time workloads. Variability introduced by dynamic systems.

In airport, forgot PC power supply

Passing through airport security, on my way to ISCA, I realized I had forgotten my work laptop PC's power supply (AC adapter). Typing this on my OLPC XO non-work computer.

I remember packing my power supply and PC, but then unpacked them to print my boarding pass. Must have forgotten to repack the power supply, although I have the laptop.

I wish PC power were more ubiquitous. One of my PCs - which,the Concerto perhaps? - had no separate AC adapter. Instead, AC right inside the latop. Harder to forget.

Error reporting in libraries

It's a perennial problem: You have written a library dfunction, perhaps a header-only class. Usually it is silent. However, occasionally it suffers errors or warnings. The interface does not provide a nice way to return an error code - e.g. you are trying to return a string, not an int where 0 or -1 indicates failure. You want to print error messages or otherwise report errors for these hopefully rare cases.

Special case: assertion failures. If you are writing something like a simulator, with simulator stdout and stderr separate from program-under-simulation stdout or stderr, you probaby want to ensure your assertion failures go the the simulator log. This is usually easy, but in Pin simulator and program file stdout and stderr are shared.

Note: we are not just talking about error messages that are followed by death. Sometikes we are talking about warning or informational messages, after which the program should keep running.

if everything belongs to a single class, you might give the class daa members for the streams to be used. e.g.

class Foo {
private:
    std::ostream m_cout;
    std::ostream m_cerr;
public:
    std::ostream& my_cout();
    std::ostream& my_cerr();
};

with the usual methods wrapping the data member ostreams.
E.g. to allow you to print to std::cout and std::cerr if not initialized.
But not everything lies in the same object/class.

Passing around ostreams to every possible function is ugly.

Adding ostreams to every possible class is ugly.

I've occasionally passed generic "env" objects to lots of functions, where the env carries things like ostreams, and other good stuff like exit functions.
But this is ugly.

My BKM is beginning to look like ... Cpp (C preprocessor) macros COUT and CERR used everywhere. With COUT defined tio be std::cout, or my_cout(), or whatever.


Formalized as a header file COUT_CERR_ostream_redirection.hpp:

// Interface:
// 
// To use std::cout and std::cerr
//   #define USE_STD_COUT 1
//   #include "COUT_CERR_ostream_redirection.hpp"
// This is also the default - if neither USE_STD_COUT and USE_MY_COUT_FUNCTIONS are defined
// 
// To use my_cout() and my_cerr()
//   #define USE_MY_COUT_FUNCTIONS 1
//   #include "COUT_CERR_ostream_redirection.hpp"
// 
// To use neither
//   #define USE_STD_COUT 0
//   #define USE_MY_COUT_FUNCTIONS 0
//   #include "COUT_CERR_ostream_redirection.hpp"
// 
// If you want to #include this header multiple times
// caller can #define INCLUDE_COUT_CERR_OSTREAM_REDIRECTION
// - equivalent to doing a #undef COUT_CERR_OSTREAM_REDIRECTION_already_included

#ifdef INCLUDE_COUT_CERR_OSTREAM_REDIRECTION
#undef COUT_CERR_OSTREAM_REDIRECTION_already_included
#undef INCLUDE_COUT_CERR_OSTREAM_REDIRECTION
#endif

#ifndef COUT_CERR_OSTREAM_REDIRECTION_already_included
#define COUT_CERR_OSTREAM_REDIRECTION_already_included

#if !defined(USE_STD_COUT) && !defined(USE_MY_COUT_FUNCTIONS)
#define USE_STD_COUT 1
#endif

#if USE_STD_COUT
#define COUT std::cout
#define CERR std::cerr
#elif USE_MY_COUT_FUNCTIONS
extern std::ostream& my_cout();
extern std::ostream& my_cerr();
#define COUT my_cerr()
#define CERR my_cout()
#else
// let user define his or her own COUT and CERR macros
// e.g. #define COUT foo_cout() in module foo, and bar_cout() in module_bar()
#endif

#endif //COUT_CERR_OSTREAM_REDIRECTION_already_included