• Esp@lemmy.blahaj.zone
      link
      fedilink
      English
      arrow-up
      38
      ·
      1 year ago

      Yeah, this same exact story keeps coming up for years now just with different names. Why anyone would think that both the ineffectiveness and racial bias in these systems either wouldn’t exist or will somehow go away eventually is beyond me. Just expensive and ineffective mass surveillance for the sake of it…

    • steltek@lemm.ee
      link
      fedilink
      English
      arrow-up
      19
      ·
      edit-2
      1 year ago

      Who remembers the HP computer that was unable to identify black people? One of my favorite “oooph, that’s not a good look” tech fails of all time. At least the people in that video were having a good laugh about it.

      https://www.youtube.com/watch?v=t4DT3tQqgRM

      Holy hell, that was 13 years ago.

    • oct2pus@kbin.social
      link
      fedilink
      arrow-up
      17
      ·
      1 year ago

      Minor correction.
      15 out of 15 requests were of black people. 14 of those requests were black men and 1 was a black woman.

      • yA3xAKQMbq@lemm.ee
        link
        fedilink
        English
        arrow-up
        16
        ·
        edit-2
        1 year ago

        Yeah, but statistics is a b*tch.

        We had a similar technology for a test run some years ago at a train station in Berlin, capital of Germany and largest city in the EU with 3.8M.

        The results the government happily touted as a success were devastating. They had a true positive rate of 80% (and this was already cooked since they tested several systems at several locations but only reported the best results), which is really not that good to start with.

        But they were also extremely proud of the false negative positive rate, which was below 0.1%. That doesn’t sound too bad, does it?

        Well, let’s see…

        True positive means you actually identified the people you were looking for. Now, I don’t know the number of people Berlin’s police is actively looking for, but it’s not that much. And the chances of one of them actually passing that very station are even worse. And out of that, you have 20% undetected. That’s one out of five. Great. If I were a terrorist, I would happily take that chance.

        So now let’s have a look at the false negative positive rate, which means you incorrectly identified a totally harmless person as a terrorist/infected/whatever. The population for that condition is: everyone passing through that station.

        Let’s assume there’s a 100k people on any given day (which IIRC is roughly half of what that station in Berlin actually has). 0.1% of 100k is 100 people, every day, who are mistakenly reported as „terrorists“. Yay.

    • Blamemeta@lemmy.world
      link
      fedilink
      English
      arrow-up
      9
      ·
      edit-2
      1 year ago

      Yeah. Basicly anything with a lower contrast, with shadows and backgrounds. And because shadows are dark, they have a lower contrast with other dark things.

    • P03 Locke@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      9
      ·
      1 year ago

      Discrimination is the wrong word. Technology has no morals or sense of justice. It is bias in the data that developers should have accounted for.

      • steltek@lemm.ee
        link
        fedilink
        English
        arrow-up
        11
        ·
        1 year ago

        It’s totally accurate though. It’s like the definition of systemic racism really. Think about housing or financial policy that disproportionately fails for minorities. They aren’t some Klan manifesto. Instead they just include banal qualifications and exemptions that end up at the same result.

      • slumberlust@lemmy.world
        link
        fedilink
        English
        arrow-up
        8
        ·
        1 year ago

        This seems shortsighted. You are basically asking people to police their own biases. That’s a tall ask for something no one can claim immunity from.

        • P03 Locke@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          1
          ·
          1 year ago

          I am asking a group of scientists who should be very well-versed in statistics and weights, you know, one of the biggest components in a machine learning model, to account for how biased their data is when engineering their model.

          It’s really not a hard ask.

      • Cortell@lemmy.world
        link
        fedilink
        English
        arrow-up
        7
        ·
        1 year ago

        Ask the people who create the data sets that machine learning models train on how they feel about racism and get back to us

      • Corhen@lemmy.world
        link
        fedilink
        English
        arrow-up
        6
        ·
        1 year ago

        It can be an imported bias/descrimination. I still think that words fair.

        Do you have a more accurate word?

        • P03 Locke@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 year ago

          I already said it: bias. It’s a common problem with LLMs and other machine learning models that model engineers need to watch out for.

      • HardlightCereal@lemmy.world
        link
        fedilink
        English
        arrow-up
        9
        arrow-down
        11
        ·
        1 year ago

        You need to learn some critical race theory. Racist systems turn innocent intentions into racist actions. If a PhD student trains an AI model on only white people because the university only has white students, then that AI model is going to fail black people because black people were already failed by university admissions. Innocent intention plus racist system equals racist action.