Tips for more effective fuzz testing with AFL++

(nullprogram.com)

73 points | by luu 4 days ago

3 comments

  • jonhohle 3 hours ago
    I used to think writing safe code in C/C++ was a matter of skill. However, after finding more than a few issues with AFL at a FAANG in code written by engineers with the highest engineering title, I changed my mind. Incredible tool.
    • fasterik 3 hours ago
      It is a matter of skill to a large extent, though. A skilled programmer will write fewer bugs on average than an unskilled one. They'll know the patterns to avoid and have a higher bar for quality. Nobody can write 100% bug-free code, independent of programming language. Also, job title might correlate with skill but it doesn't rule out incompetence at the highest levels.
      • tikhonj 3 minutes ago
        Writing safer C is a matter of skill.

        Writing safe C is a matter of either radically different (much, much slower) process or better tooling or both. Skill is not sufficient.

      • kragen 2 hours ago
        Sometimes writing shitty code is a result of competence, not incompetence. I've seen much better programmers than I ever will be writing terrible code because it was the fastest way to get a prototype running and find out whether the problem was worth writing good code for.
      • ambicapter 2 hours ago
        It doesn't really matter, the most skilled programmers are usually working on harder stuff, so the bug rate remains fairly consistent.
      • Neywiny 3 hours ago
        I think it's mostly a testing thing. Valgrind is a friend a lot of people haven't heard of for c/c++. Setting up good regression tests is an art. Maintaining them is even harder.
    • PorciiVorbesc 2 hours ago
      >I used to think writing safe code in C/C++ was a matter of skill.

      It's a matter of how many fucks the devs gave about doing quality work. You can find inexperienced people deliver quality work simply because they're passionate and they care (Half-Life was made in part by people with no prior game or coding experience), and you can find supposed rockstars deliver slop because for whatever reason they don't care, they just want to punch in, collect paychecks, then punch out.

      FAANG stopped being a benchmark of skill a while ago. It's a skill of griding leetcode and 7 stage interviews to get it, then resting and vesting till you can jump ship to something better or get mass layd off.

      • pfdietz 1 hour ago
        > It's a matter of how many fucks the devs gave about doing quality work.

        It's more a matter of how much management cares about quality, and how much time and how many resources they're willing to devote to it.

        And that's a function of how much the quality of the code actually matters in the market.

        • PorciiVorbesc 1 hour ago
          FANG doesn't have resources, but indie coders and game studios do?
          • nxobject 41 minutes ago
            I think you’re a bit optimistic about the quality of a triple-A game’s codebase that was rushed to a holiday release date. I have slightly more tolerance for bugginess in a game.
          • pfdietz 1 hour ago
            Notice I said how much management is willing to devote, not how much they have.
      • sczi 32 minutes ago
        Has there ever existed a C or C++ codebase of significant size that hasn't had exploitable memory corruption issues? I don't think we can keep saying it's a question of the dev's passion or skill or whatever, if no dev has ever managed to do it.

        That's not to say memory corruption issues are the only kind of issues, and rewriting a mature C/C++ codebase into a memory safe language may well introduce more issues than it fixes...

  • monster_truck 4 hours ago
    These are good tips, thank you
  • botquests 4 hours ago
    [flagged]