When talking about your time travel though experiment, I realized that you probably had no course about military usage, ethics and the risks of our work being used for nefarious purposes, just like I had none and just like almost everyone else had none. I don't know if it's better nowadays but I doubt it. It's a shame because we only need a few warnings; something along the lines of "your code can be reused easily without you knowing, it can be ported, added and modified for purposes you're not aware of".
This also reminded me of a project the consulting company I worked at years ago (my first job) received a project which was basically getting a video feed and displaying it along with additional stuff. With context, you know where it's going but without hints, it's very generic. Funnily enough, this genericity was extreme and it was very difficult to get details about other components that interfaced with this; it became obvious this was a military project in disguise, most probably a sealed armored vehicle with cameras outside to provide vision to the people inside it. In the end, the project was turned down (also partly because hiding information makes for poor commerical relations).
My university has just introduced a mandatory ethics module for Stage 1 students, whereas it used to be an elective before. I'm two years too late to have done the new module, and didn't take the elective when it was available (in favour of Creative Writing), but I'm glad they've made this change now.
I only wish that I knew to read this when it released. I feel like my bachelor's (which started in 2017) was almost specifically designed to make students not question what they were working on. Had to learn this lesson myself, almost exactly the same way you did.
This is an anecdote about ethics when working as a software engineer. There was no ethics tag, so I thought philosophy was the closest.
I read this quite a while ago and it's haunted me ever since. Even though the author's experience was working for a military contractor, I like how they draw parallels to software engineering in general.
I get the same feeling but at the same time, there has been enough willing humans for these tasks for a long time. Trump-Musk's DOGE was also a good example of how a few (also young) people will do it, maybe with a lot of enthutiasm.
We've had couple cars with SDRs and jammers in there. One of the things that kept happening was that the enemy burried improvised explosive in the dirt road and detonated it by calling the attached phone as a convoy went over it. Jamming phone bands and WiFi helped a lot. And obviously direction finding SDR...
When I say we, I mean Czechia. I am not sure sure how I feel about that. Mixed feelings, probably? That's the issue with tech, usually. And geopolitics.
One thing I find difficult at work is to evaluate how ethical or unethical something is. Even if I determine what the "worst use case" of the product is, how do I determine how ethical it is? Is automation ethical (it puts people out of work)? Is work in aircraft development ethical (aircraft are a driver of climate change)?
The last time I was actually taught ethics was in high school, and that was basically an introduction, and also I had no idea whether or how it would apply to me in real life. My software development curriculum did not include any ethics course, unfortunately.
Can you recommend good resources to learn about this kind of "applied ethics in software development"?
Working with drones has a lot of this. They are almost by definition devices that are very very easy to turn into a missile. I thought about that a lot, even before the Ukraine war started.
I'm just happy that I work at a company that skirts the line sometimes but is generally hell-bent against making weapons. As it is, we don't "find anything" besides flat ground to land on, "evade anything" that isn't a tree or power line, or "deliver anything" that doesn't have someone on the other side to willingly pick it up. Even if that someone is sometimes a soldier. :-/
calebhearth | 18 hours ago
Oh hey looks like this is making the rounds again. OP here, glad to see this on the front page.
adrien | 12 hours ago
It's a good piece.
When talking about your time travel though experiment, I realized that you probably had no course about military usage, ethics and the risks of our work being used for nefarious purposes, just like I had none and just like almost everyone else had none. I don't know if it's better nowadays but I doubt it. It's a shame because we only need a few warnings; something along the lines of "your code can be reused easily without you knowing, it can be ported, added and modified for purposes you're not aware of".
This also reminded me of a project the consulting company I worked at years ago (my first job) received a project which was basically getting a video feed and displaying it along with additional stuff. With context, you know where it's going but without hints, it's very generic. Funnily enough, this genericity was extreme and it was very difficult to get details about other components that interfaced with this; it became obvious this was a military project in disguise, most probably a sealed armored vehicle with cameras outside to provide vision to the people inside it. In the end, the project was turned down (also partly because hiding information makes for poor commerical relations).
[OP] seafoamteal | 3 hours ago
My university has just introduced a mandatory ethics module for Stage 1 students, whereas it used to be an elective before. I'm two years too late to have done the new module, and didn't take the elective when it was available (in favour of Creative Writing), but I'm glad they've made this change now.
addison | 11 hours ago
I only wish that I knew to read this when it released. I feel like my bachelor's (which started in 2017) was almost specifically designed to make students not question what they were working on. Had to learn this lesson myself, almost exactly the same way you did.
[OP] seafoamteal | a day ago
This is an anecdote about ethics when working as a software engineer. There was no ethics tag, so I thought philosophy was the closest.
I read this quite a while ago and it's haunted me ever since. Even though the author's experience was working for a military contractor, I like how they draw parallels to software engineering in general.
singpolyma | 21 hours ago
And now we've made the tools so that when every dev says "no" they can just ask the LLM to do it...
adrien | 12 hours ago
I get the same feeling but at the same time, there has been enough willing humans for these tasks for a long time. Trump-Musk's DOGE was also a good example of how a few (also young) people will do it, maybe with a lot of enthutiasm.
mordae | a day ago
We've had couple cars with SDRs and jammers in there. One of the things that kept happening was that the enemy burried improvised explosive in the dirt road and detonated it by calling the attached phone as a convoy went over it. Jamming phone bands and WiFi helped a lot. And obviously direction finding SDR...
When I say we, I mean Czechia. I am not sure sure how I feel about that. Mixed feelings, probably? That's the issue with tech, usually. And geopolitics.
doctor_eval | an hour ago
Did you mean Chechnya?
Riolku | 16 hours ago
This is a chilling read. Thank you for sharing
landon | 10 hours ago
Yeah... I miss the age when people had enough hope to write things like this.
spenc | a day ago
I love this article so much <3 Feels especially relevant for me to reread now as I go into a new job at Microsoft
oger | 5 hours ago
Great post, thanks for writing this!
One thing I find difficult at work is to evaluate how ethical or unethical something is. Even if I determine what the "worst use case" of the product is, how do I determine how ethical it is? Is automation ethical (it puts people out of work)? Is work in aircraft development ethical (aircraft are a driver of climate change)?
The last time I was actually taught ethics was in high school, and that was basically an introduction, and also I had no idea whether or how it would apply to me in real life. My software development curriculum did not include any ethics course, unfortunately.
Can you recommend good resources to learn about this kind of "applied ethics in software development"?
icefox | 2 hours ago
Working with drones has a lot of this. They are almost by definition devices that are very very easy to turn into a missile. I thought about that a lot, even before the Ukraine war started.
I'm just happy that I work at a company that skirts the line sometimes but is generally hell-bent against making weapons. As it is, we don't "find anything" besides flat ground to land on, "evade anything" that isn't a tree or power line, or "deliver anything" that doesn't have someone on the other side to willingly pick it up. Even if that someone is sometimes a soldier. :-/