Is this ethical? #480
Replies: 4 comments 13 replies
-
|
I am not connected to the deep-live-cam project. Your concerns are valid and you might be interested in following the $500,000 NEH grant (United States) to study these issues, announced the day after you wrote your comment. https://chass.ncsu.edu/news/2024/08/28/nc-state-secures-500k-neh-grant-to-launch-ai-center-in-chass/ The only benefit I see with this deepfake technology is to continue to ensure females of all ages look "perfect" online, as in Jane Jetson's "morning mask" (e g., link below). You know this type of live face swapping software will bring in a lot of money, not just be used to screw up elections, wreck relationships, and incite wars. All the big make-up companies will have their own app! Of course there will be more screwed up kids, but who cares. (BTW I'm being facetious.) https://youtu.be/EwOcT3sK8Eg?feature=shared Good luck. This post will likely be removed so I hope you get this. |
Beta Was this translation helpful? Give feedback.
-
|
If you’re right and the authors are deleting discussions about ethicality (which is WILDLY unethical) then we should stop discussing it and start taking action to have this repo removed, just as a start. |
Beta Was this translation helpful? Give feedback.
-
|
I never deleted any threads about ethics on this. And I am very aware you started this 2 days ago. |
Beta Was this translation helpful? Give feedback.
-
|
I’m relieved to hear you’re not deleting posts; however, you still haven’t addressed any of my concerns despite being aware of them since I made the post. |
Beta Was this translation helpful? Give feedback.
-
|
The reason this was publicly made is for the awareness of everyone that this is possible. About the use cases, there are tons especially on the entertainment industry as well as discriminations and others as well. This has been available for more than a year but only gained traction just this month. I really don't want to answer this since you already put a conclusion so I leave the answer for the community who use this. Since ethics all ends up to the user and not the application itself. It's like saying torrent is bad, as well as tor and other tools being used maliciously aside from their proposed use case. Either way, hoping you learn something from this tool. And hoping other experts learn from this tool as well (knowing this exist) and create a tool to prevent abuse on this kind of software. |
Beta Was this translation helpful? Give feedback.
-
|
Others have already replied to my concerns directly while you continue to dodge them. Trying to shirk responsibility by saying “ethics all ends up on the user” is irresponsible; ethics should be a concern at every step of the process, a process that heavily involves the author, i.e. you. With that said, I will ask one more time: I’ve read already about your steps to combat porn and gore and other explicit media, but what about misinformation? What steps have been taken to combat that? Why are you willing to mitigate some misuse, but not all of it? Following up, you’ve given me the use cases outlined in the readme, which again, I’ve already read, yet still have not commented on the ethicality of those cases either. For example, when Disney used deep fakes for Mark Hamill and CGI’d Carrie Fisher after her death, it brought up a lot of discourse on the entertainment industry about “immortalizing” someone for profit, especially since there really aren’t any laws surrounding this. Could you give me an example of this being used ethically? As for your point on why you released this, I understand that it’s important to make the world aware that this is possible; however, you have also made it accessible, which is very different. Awareness can be raised in so many other ways like papers, articles, social media posts, or even getting in touch with security or AI researchers directly and letting them deal with it. |
Beta Was this translation helpful? Give feedback.
-
|
There is no doubt this technology will perpetually be used for unethical purposes, perhaps the most disturbing of these being pornography. In other words, if you don't know how easy it is to create deep fakes you are more vulnerable to its influence, which makes its use as a psychological weapon of control much more effective. So I think it's a good thing to know that it is possible to do! That being said, it will take copious discussion, mountains of research, and many brave heroes to combat the monstrous egregore of pornography this technology will most certainly invigorate... I cannot lie though that I am very happy I spent half of my day head scratching over errors, figuring out how to install this thing. I am happy because I was able to get it working! I think what makes it acceptable to use though, is that I do not wish to use this tool to engross myself in a delusion of self-desires. Like many others, I will wield this as a tool for art—to express the uncanny and zany memes of our modern era, which is very much driven by these novel technologies becoming easily accessible to everyday people. We are playing with a Promethean fire, and should fire cease to exist for the fact that it burns the ignorant? |
Beta Was this translation helpful? Give feedback.
-
|
我是个计算机菜鸟，冒昧地请教您，请问您是如何打开这个程序的呢？我只下载了压缩包，但并不能使其运行，原谅我才疏学浅，这些新奇的东西对我而言确实是不知如何是好。 但就伪造技术是否合乎道德这个话题，我觉得关键还是使用者的心理是否合乎道德。时代在进步，新事物的出现是无人可挡的，即便没有这位大佬贡献开源，也会有其他人出现。我觉得这种新奇的公开本身意味着道德，如果她仅仅被少数人掌控，我不敢想象。工具永远是无罪的，菜刀可以杀人，难道我们就要销毁天下所有的菜刀吗？那杀人的菜刀是金属做的便要管控天下所有的金属吗？这种对工具是否合乎道德的讨论本身就是一种道德绑架！客体是客观存在的，是无法使其销毁的，我们能改变的是使用者这个主体，但主体通常是不可抑制的，这关系到利益和人性。但我们不能因为一种工具的出现可能带来坏处就抑制不前，楼主出发点是好的，但是也高看了作者和工具的作用，这是个社会问题，而不是作者删除开源就能解决的问题。 不过有没有人可以教教我怎么运行这个程序? |
Beta Was this translation helpful? Give feedback.
-
I have to run translate to understand this but I would suggest following the instruction on the git will be the most straightforward solution on what you are trying to ask. If you don't have any knowledge on how to do that, I would suggest getting the gumroad edition. It's simpler and easier to run (basically a package) and it helps us pursue further on how to improve this app. |
Beta Was this translation helpful? Give feedback.
-
|
Since it’s been two weeks and @hacksider still has not addressed my points, I’m going to assume he does not have the examples I asked for and would like to reiterate for anyone else who sees this: what are the ethical use cases for this repo? |
Beta Was this translation helpful? Give feedback.
-
|
they are already disabling related apps. Like the one i used - Roop unleashed. https://github.com/C0untFloyd/roop-unleashed I’m pretty new to GitHub, so I initially posted this in the wrong place. I found this issue topic later, but I still think this is a really bad direction to take. We need to learn how to deal with the dangers and possibilities that come with the level of AI we have today. Right now, it’s a Python Git project, and it’s about 90% perfect, but most people can still tell the difference between fake and real. Removing apps like this from a platform as big as GitHub at this point means our society doesn’t have to confront the consequences right now, while only a handful of people have the knowledge to use this technology in harmful ways. In a few years, the entire process of creating deepfakes and face-swaps will likely be indistinguishable from reality. By then, it’ll be as simple as using an AI-powered Android app on your iPhone 25. It would be smarter to allow a small percentage of people to use it now, so we can create new laws, develop detection tools, adapt, and raise awareness about its possibilities and risks. Disabling these apps right now just means the problem will be much bigger in the future when AI makes it even easier to misuse this technology. Imagine cave people inventing the hammer, 1 cavemen kills another cavemen. So they decide to trow all hammers away. and to not tell their children what a hammer is, and how to use it safely. And that its a bad idea to hit someone with a hammer. 50 year later, when the children have kids themself. There is a big hammer rain, and there is hammers everywhere. No info to be found about this strange tool. Because when they search for hammer instructions on github it says This repository has been disabled. |
Beta Was this translation helpful? Give feedback.
-
Hey, check the README for a few of the many possible use cases of this tech. |
Beta Was this translation helpful? Give feedback.
-
|
Literally the third sentence of my original post:
Why is it so difficult to actually read what I am writing and answer me directly? I genuinely don't understand. And for the record, I re-read the README before I posted this reply. The disclaimer has barely changed and still makes no mention of the issues I've brought up in this discussion. Elon Musk is still the lead example. |
Beta Was this translation helpful? Give feedback.
-
I think you need to re-read the readme again... both the disclaimer and features & uses. |
Beta Was this translation helpful? Give feedback.
-
|
I work in a nursing home. One of the elderly ladies (88, in the early stages of dementia) really missed her late husband. They had been married for 65 years. I found some pictures of him and combined them with old footage of an elderly man on YouTube who had a similar posture. Then, I uploaded them to her virtual photo frame. She cried and cried. The next month, I took it a step further. I wrote a script for Roop-Unleashed to swap 180-degree 3D SBS frames, adding slight morphing to the face landmarks. Then, I took a few scenes from The Intern (featuring an older Robert De Niro) and replaced his face with her late husband's. She had never heard of VR before and was a bit scared when I showed her the headset. But after some tweaking and explaining what was about to happen, she gave it a try. I had only made about 2–3 minutes of content, but she kept asking me to play it over and over until my Quest VR headset's battery died. Of course, there’s the argument: "You put a face on a copyrighted movie, blah blah blah." But what I saw was a unique service—something nobody else on earth is providing right now. And there are so many potential use cases for this. My next step would be to film or generate unique experiences—kind of like those "girlfriend experience" VR videos, but without the sexual aspect. Just simple, universal moments, like a man walking beside you, holding your hand on the beach, or a woman doing the same. With AI, you could generate hundreds of these basic scenes, then personalize them by adding someone's face. How many people lose a child too young? How many lose their partner to cancer far too soon? Some people find ways to cope, some can’t even look at a picture of their loved one anymore, and some would truly benefit from custom VR videos like this to keep their memories alive. This is what VR was made for. Seeing someone almost eye to eye in a 3D VR video is a completely different experience from looking at a photo or watching a video on a phone screen. |
Beta Was this translation helpful? Give feedback.
-
|
Great work @Elise96nl , would love to reach out to you... |
Beta Was this translation helpful? Give feedback.
-
The simple fact that even the examples show this tech being used with famous and political figures is deeply disturbing. I have already searched through the discussions with no results found for ethics. I have also read the statement in the README and I understand that you are taking “precautions” to prevent misuse, but I don’t think it’s enough. Your statement doesn’t even mention the misinformation or scam potential, which is also disturbing, especially after tech nearly identical to this was used in a $25M scam in Feb 2024. This repo has, essentially, taken technology that was previously only accessible to the most skilled and resourceful scammers and placed it in the hands of script kiddies. I fail to see how a repo like this has ANY ethical application whatsoever and I believe that even open sourcing this application was unethical.
Beta Was this translation helpful? Give feedback.
All reactions