Oxide computer 3D rack guided tour

(explorer.oxide.computer)

112 points | by darthcloud 3 days ago

16 comments

  • jjice 1 hour ago
    I want Oxide to do so well. The product is a breath of fresh air in the era of cloud providers. As an engineer, I'd kill to get to work with their hardware.

    Not to mention that working at Oxide sounds like a modern Sun Microsystems with the ideology that team has. Highly recommend their podcast "Oxide and Friends", and their original "On The Metal" show.

    I've attempted to apply to their company multiple times over the years, only to be stun locked by the application process. Not because it's a bad process, but because I feel I'm not up to par as an engineer. Maybe one day I'll go through with it.

    • EvanAnderson 32 minutes ago
      Oxide certainly sounds cool. It reminds me of when I dealt with DEC gear back in the 90s. That stuff felt more like "real computers" than any of the IBM PC-derived drek I'd worked with.

      I don't need to work there-- I just wish I could work with the Oxide gear in Customer engagement, too. I don't work with businesses big enough to need it, sadly. It looks so sweet.

    • Melatonic 10 minutes ago
      Seriously - looks so cool
    • thraway3837 44 minutes ago
      Just a gentle reminder that a company may portray itself as cool to customers, but is not cool to their own current or future employees.

      Their interview process was shady. There was a post here about 1-2 years ago that was a link to their interview process and how open and transparent they were. The post itself was from an employee and a fellow commenter who was gaslighting folks was also an employee. Several folks complained about the tremendous amount of homework they had to do after the initial screen, and once submitted, were ghosted. One of employees repeatedly rebutted that claim in the comments, and they did this for quite a few commenters. Was a not a good look. I doubt much has improved since then as seeing the comments below confirms the same mess.

      Don't spend time being amazed by folks who won't treat you right. It just ain't worth it.

      • marshray 1 minute ago
        I'm seeing the phrases "tremendous amount of homework", "substantial amount", and "few hours".

        Does anyone have an actual estimated time we can discuss?

      • dijit 33 minutes ago
        Devils advocate (really not affiliated with oxide, but I have worked for a “desirable” employer before).

        How would you handle a few thousand applicants for a single role?

        I think no matter what you do it will feel inhumane, we can argue that a few hours of work for a take home test is inhumane too, being ghosted after doing one definitely wouldn’t pass my personal bar of acceptability, but if its the first stage and the task would take a properly qualified applicant less than 30 minutes then I can’t fault.

        How would you do things? remember that it has to scale and you cant leave any gaps based on human fallibility (HR/Hiring Managers are humans and will forget if there are too many things going on at once).

        • throwaway219450 13 minutes ago
          If you truly believe you’re “scaling” you do it the Google way and have a strict loop with a good rubric for the interview so applicants are comparable. The whole point of that system is thousands of people and hundreds of interviewers, and a very standard process. I’ve always found it pretty fair even with some randomness in scoring.

          You shouldn’t be giving take homes unless they’re either short, or the applicant passed a screen and you’re investing time. Otherwise how are you “scaling” the review? Claude? Hidden test suite (not bad)? Some sort of leaderboard (bad, rewards people with time), something else?

          • dijit 11 minutes ago
            I’ve been through the Google process and I don’t think I would consider it to be the opposite of inhumane.
            • DANmode 4 minutes ago
              Use less negatives.
              • dijit 0 minutes ago
                it was intentionally this way
        • jaggederest 22 minutes ago
          There's a simple answer, if someone is doing a substantial amount of work for your interview process, pay them an amount of money that is more than zero but less than "do job interviews for a living". Or provide that amount times two to a charity of their choice.

          I've done this for hiring before, for people who reached the "put substantial effort in" stage (in my case basically 2nd or 3rd round work sample stuff), and it was a great way to make sure we got good signal and they felt respected.

          • dijit 1 minute ago
            “put substantial effort into it” is such a personal thing.

            DDG hires like this, actually, and if I recall correctly I would be paid a flat fee, it would take a week, and the work I did would be part of something genuine in DDG, maybe a bug or something.

            Now, that probably sounds good to you, but taking a week out of my current employment is not going to happen- there’s an incentive to go “over the hours” inherent to the ask, even if you’re paying me a flat rate, I might lose to someone equally qualified who puts in 1.01n into the task, so I should put 1.02n (etc; ad infinitum).

            Which is part of the issue with all take home assignments. I have given out take home assignments (given to HR to be administered) which should take a qualified candidate 20 minutes to finish beginning to end (as in, including syncing the project, setting up their editor, exploring the problem, googling around about things, trying it out and then following up with the email to HR). I don’t doubt for even a moment that someone has spent several hours on this problem- because they’re not qualified.

            Passing the HR barrier in that case will not help them unfortunately, because they’ll get to talk to me, and I will disqualify them in all likelihood, and candidates are told that it should take not more than a half hour, but en masse: people don’t listen.

            The trouble is, theres thousands of applicants, a handful of HR, and one me.

            Not to be on some kind of pedestal (I’m not), but the problem doesn’t scale, you need only apply the tiniest amount of systems thinking to see it.

          • fred_is_fred 18 minutes ago
            From a legal and financial perspective it seems like it would be difficult to pay people to do interview homework. There's tax implications and other issues like state labor laws.
            • icedchai 9 minutes ago
              People do contract / temporary / 1099 work all the time. It's very simple.
    • convolvatron 1 hour ago
      I've gone through the same process, not so much that I don't think I would be worth considering, but serious code and documentation examples aren't something I can really give out given that they're proprietary. this last winter I started a whole guest-kernel based syscall intermediation and distribution framework in rust just for the application. with all kinds of design documents. I was about 30% finished by the time I landed a job somewhere else :)

      but I still applaud the intent. I self-selected out by giving into scope creep

      • TZubiri 1 hour ago
        I'd be interested in the context if you'd be willing to share.

        It sounds from the outside like Oxide has an interview process that requires some low level engineering work to be delivered? Maybe I got that wrong.

        • convolvatron 1 hour ago
          no, they want a questionnaire, a coding sample, and an example of technical writing. there's a reasonable interpretation of that that doesn't involve writing a distributed unix.
          • TZubiri 1 hour ago
            Sounds approachable, and something that would be evaluated based on merit.

            As usual, I'm assuming the assignment is evaluated based on a reasonable time-commitment. From what the recruiting experts tell me, it's a good strategy to spend as much time as possible, the deliverable is better, and the optics aren't bad either, it signals investment into the application instead of signalling spray and pray application broadcasting.

  • bradfa 24 minutes ago
    Curious how robust the (what looks like PCIe edge connector slots) connection to the drives is in practice. Obviously converting from the horizontal mainboard to a vertical drive requires such a connection, making it a plug-in card at least allows for replacing the card if it breaks/wears/etc, and mounting the front of the adapter card to a bulkhead should prevent much shifting of the card in the slot. Neat design and reuse of a cheap high speed connector.
  • dcre 35 minutes ago
  • bradfa 18 minutes ago
    How easy is it to swap the fan bar out for a failed fan? It looks like a single unit holding all the fans. Can the sled be pulled but retained in the rack and then fan bars removed and reinstalled without fully removing the sled and without tools?
    • sudomateo 11 minutes ago
      The sleds blind mate to power and network on the backplane so removing the sled will disconnect it. Once the sled is disconnected you remove its airflow shroud and pull the fan assembly out.
  • andrewl-hn 1 hour ago
    This design feels very obvious-in-hindsight. Consolidate power adapters and networking, replace cabling with pluggable slots. It's something similar to what IBM mainframes or Sun cabinets could've looked like. Somehow hardware giants like Dell, HP, SuperMicro, etc didn't make a product like this, even at their peak in 2000s or during cloud boom in 2010s. I wonder why?

    Beautiful machine, and fun to see Illumos heart still beating inside!

    • bri3d 49 minutes ago
      > Somehow hardware giants like Dell, HP, SuperMicro, etc didn't make a product like this, even at their peak in 2000s or during cloud boom in 2010s.

      Not so sure about this one. HCI (Hyperconverged) rack units (where storage and compute live in the same racked systems) and "blade servers" have been a thing for a really long time now; compute sleds aren't what's novel here.

      Rack-level DC conversion is also not particularly novel, although underutilized IMO. It was pretty popular in HPC style density applications for awhile (see HP/SGI Altix 4000 for a good old example).

      What's unique about Oxide is that they went all the way down to the firmware and then back up, rather than doing commodity hardware integration or reselling - for example, you can get something like a Supermicro EVO:Rail, but it will be running VMWare, not a fully integrated platform.

    • steveklabnik 3 minutes ago
      The big difference that everyone is missing in this subthread is that Oxide is about the hardware and the software.

      There are systems which have similar overall hardware designs, but they are usually integrating a large amount of hardware and software from multiple vendors. Oxide is much closer to "everything is produced by Oxide."

      I wrote this back in 2022, and it's still fundamentally relevant today: https://news.ycombinator.com/item?id=30678324

    • baby_souffle 44 minutes ago
      > Somehow hardware giants like Dell, HP, SuperMicro, etc didn't make a product like this,

      Dell and HP both have "blades" that plugged into a blade-chassis. The chassis had all the lights out mgmt as well as power/networking integrated so the blade was basically a metal box with compute/memory/storage and it just slid in to the dock.

      I am sure that supermico had something like this as well

      • Melatonic 7 minutes ago
        Cisco does too and theres another hardware virtualization layer below the normal ones ( so for example you can have many virtual nics per actual nic, etc)
    • wmf 55 minutes ago
      The vendors did make blades in the 2000s.
      • creaturemachine 20 minutes ago
        I thought surely this isn't just blade servers, that those compute shelves were full of GPUs or something novel, but no just blades reincarnated. I used to support HP's baby version of this, the c3000.
      • jeffbee 36 minutes ago
        Also, a big cabinet into which you plug varying amounts of hardware capacity, then use the control plane to partition into various virtual resources, describes at least at the conceptual level IBM going back decades.
    • fred_is_fred 17 minutes ago
      > Somehow hardware giants like Dell, HP, SuperMicro, etc didn't make a product like this, even at their peak in 2000s or during cloud boom in 2010s. I wonder why?

      They all did. HP had Super Dome and blades and Synergy. Dell had similar.

    • manwithopinions 39 minutes ago
      I learned about blade servers back in ~2010 because Blizzard used to run World of Warcraft realms on them and auctioned them off for charity.

      https://warcraft.wiki.gg/wiki/Server_blade

      https://en.wikipedia.org/wiki/Blade_server

  • girfan 28 minutes ago
    Cool tour. I haven't kept up with their developments; what kind of workloads have they been pushing for? Since they don't seem to have any specialized accelerators in the Compute Sled, I am assuming they are not targeting AI workloads for now?
  • dralley 53 minutes ago
    It's a shame they missed out on the AI server boom.
  • kevinrineer 18 minutes ago
    I'm a bit surprised that this kind of incredibly engineered hyperconverged rack system isn't doing some form of liquid cooling.
    • schainks 1 minute ago
      Not worth the capex to do that. Air cooling with evaporators up front is pretty standard and works great, not to mention fewer moving parts.
    • wmf 8 minutes ago
      99% of data centers do not have liquid available to the rack. Liquid within the rack doesn't really help.
  • Kostic 34 minutes ago
    A very beautiful website and a machine. Oxide folks should be proud, you can see the love that went into it.
  • arjie 1 hour ago
    Oh wow, they're on the Epyc 9005 series now. Very cool. Dude, what a monster of a machine that rack is haha. Bloody hell.
  • dorongrinstein 59 minutes ago
    I met Bryan Cantrill the CEO of Oxide many years ago. He's awesome. I am rooting for Oxide to become as big as Dell.
  • PoignardAzur 1 hour ago
    The screen goes black for me after ~5s. I'm using Firefox on Linux, probably something to do with that.
  • burnrate 1 hour ago
    I wonder how much a rack like this costs
    • LoganDark 1 hour ago
      IIRC I heard that a single rack is upwards of $600k–$1m, but that was before the AI boom/crisis.
      • TZubiri 1 hour ago
        I don't recall if the price was confidential, but I will say that it's higher than that, and that it has been influenced by the RAM cost increase.

        If I recall when comparing to competition, it was premium priced, for sure, but it's more that it's so dense that you had to compare 1 Oxide rack to like 4 commodity racks. Spec for spec I recall that the premium for the verticality wasn't that high.

    • jeffbee 1 hour ago
      With only 288 128GiB DDR5 ECC DIMMs? I'm sure that doesn't cost much.
  • joshAg 44 minutes ago
    may the Sun never set
  • DiabloD3 1 hour ago
    Thats neat.
  • RobLach 1 hour ago
    Neat