Log out
My profile and settings
My bookmarks
Comment history
Please complete your account verification. Resend verification email.
today
This verification token has expired.
today
Your email address has been verified. Update my profile.
today
Your account has been deactivated. Sign in to re-activate your account.
today
View all newsletters in the newsletter archive
today
You are now unsubscribed from receiving emails.
today
Sorry, we were unable to unsubscribe you at this time.
today
0
0
Back to profile
Comment Items
You have not left any comments yet.
title
in reply to
name
description
Saved Posts
You haven’t bookmarked any posts yet.
“We need thousands of scientists following all paths that might lead us to a clean energy future.”
learn more
Become a Gates Notes Insider
Sign up
Personal Information
Title
Mr
Mrs
Ms
Miss
Mx
Dr
Cancel
Save
This email is already registered
Cancel
Save
Please verify email address. Click verification link sent to this email address or resend verification email.
Cancel
Save
Address
Cancel
Save
Email and Notification Settings
Send me updates from Bill Gates
You must provide an email
On
Off
Send me Gates Notes survey emails
On
Off
Send me the weekly Top of Mind newsletter
On
Off
Email me comment notifications
On
Off
On-screen comment notifications
On
Off
Interests
Select interests to personalize your profile and experience on Gates Notes.
Saving Lives
Energy Innovation
Improving Education
Alzheimer's
Philanthropy
Book Reviews
About Bill Gates
Account Deactivation
Click the link below to begin the account deactivation process.
If you would like to permanently delete your Gates Notes account and remove it’s content, please send us a request here.
Autopilot on steroids
When ballistic missiles can see
This book educated me about the pros and cons of autonomous weapons.
|
0

When I was a kid, I read a lot of sci-fi books. One of the most common themes was “man vs. machine,” which often took the form of robots becoming self-aware and threatening humanity. This theme has also become a staple of Hollywood movies like The Terminator and The Matrix.

Despite the prevalence of this theme, I don’t lose any sleep worrying about this scenario. But I do think we should spend more time thinking about the implications—positive and negative—of recent progress in artificial intelligence, machine learning, and machine vision. For example, militaries have begun to develop drones, ships, subs, tanks, munitions, and robotic troops with increasing levels of intelligence and autonomy.

While this use of A.I. holds great promise for reducing civilian casualties and keeping more troops out of harm’s way, it also presents the possibility of unintended consequences if we’re not careful. Earlier this year, U.N. Secretary General António Guterres called global attention to these threats: “The weaponization of artificial intelligence is a growing concern. The prospect of weapons that can select and attack a target on their own raises multiple alarms…. The prospect of machines with the discretion and power to take human life is morally repugnant.”

Unfortunately, my first attempt to educate myself on autonomous weapons was a bust. I read a book that was dry and felt really outdated. Then a few months ago I picked up Army of None: Autonomous Weapons and the Future of War, by Paul Scharre. It’s the book I had been waiting for. I can’t recommend it highly enough.

Scharre is a great thinker who has both on-the-ground experience and a high-level view. He’s a former Army Ranger who served four tours of combat duty in Iraq and Afghanistan. He then went onto a policy role at the U.S. Department of Defense and led the working group that drafted the government’s policy on autonomous weapons. He’s currently a policy expert at the Center for a New American Security, a center-left think tank in DC.

He is also a good writer. Scharre writes clearly about a huge range of topics: computer science, military strategy, history, philosophy, psychology, and ethics. He gives you the right grounding to start participating in the debate over where our country should draw the line on these powerful technologies.

Scharre makes clear from the beginning that he has no problem with some well-bounded military uses of autonomy. For example, he brings you along for a tour of the U.S. Navy’s Aegis Combat System, an advanced system for tracking and guiding missiles at sea. Aegis has a mode of operation in which human operators delegate all firing decisions to an advanced computer (but can override them if necessary). Why would you want to put a computer in charge? If you’re out at sea and an enemy fires 50 missiles at you all at once, you’d be very happy to have a system that can react much faster than a human could.

“Autonomy has great benefits in environments where humans can’t survive.”

Army of None also shows that autonomy has great benefits in environments where humans can’t survive (such as flight situations with high G forces) or in which communications have broken down. It can be enormously helpful to have an unmanned drone, tank, or sub that carries out a clear, limited mission with little communication back and forth with human controllers.

In addition, autonomous weapons could potentially help save civilian lives. Scharre cites robotics experts who argue that “autonomous weapons … could be programmed to never break the laws of war…. They wouldn’t seek revenge. They wouldn’t get angry or scared. They would take emotion out of the equation. They could kill when necessary and then turn killing off in an instant.”

Despite these and other advantages, Scharre does not want the military ever to turn over judgment to computers. To make his case, he offers compelling real-life cases in which human judgment was essential for preventing needless killing, such as his own experiences in Afghanistan. “A young girl of maybe five or six headed out of the village and up our way, two goats in trail. Ostensibly she was just herding goats, but she [was actually] spotting for Taliban fighters.” Scharre’s unit did not shoot. Yes, it would have been legal, but he argues that it would not have been morally right. A robotic sniper following strict algorithms might well have opened fire the second it detected a radio in her hand.

Scharre ends the book by exploring the possibility of an international ban on fully autonomous weapons. He concludes that this kind of absolute ban is not likely to succeed. However, he holds out hope that enlightened self-interest could bring countries together to ban specific uses of autonomous weapons, such as those that target individual people. He also believes it’s feasible to establish non-binding rules of the road that could reduce the potential for autonomous systems to set each other off accidentally. He also believes we could update the international laws of war to embed a common principle for human involvement in lethal force.

There are no easy answers here. But I agree with Scharre that we have to guard against becoming “seduced by the allure of machines—their speed, their seeming perfection, their cold precision.” And we should not leave it up to military planners or the people writing software to determine where to draw the proper lines. We need many experts and citizens across the globe to get involved in this important debate.

Read this next
NEXT
Comments
posting ...
Please verify your email in order to make comments. Click here to resend verification email
Sorry, duplicate comments are not allowed. 
Sorry, that HTML is not allowed. 
Sorry, something went wrong. 
In order to comment you must be a Gates Notes Insider. Please sign up or log in to continue. 
Be the first to leave a comment.
Comment Locked
Comments more than 2 months old are locked. For more information, contact us.
Report
Delete Comment?
Deleting this comment will remove replies to this comment by you and others as well. This action cannot be undone.
Delete Comment
Why do you want to report this comment?
It's annoying or not interesting
It's abusive and/or vulgar
It's spam
Report Comment
Your report has been submitted.
Close
Save
Cancel
Thanks for visiting the Gates Notes.
We'd like your feedback. Yes, I'll take the survey No thanks
Become a Gates Notes Insider
Join the Gates Notes community to access exclusive content, comment on stories, participate in giveaways, and more.
SIGN UP
Already joined?
Log in
Logout:


Become a Gates Notes Insider
Become a Gates Notes Insider
Join the Gates Notes community to access exclusive content, comment on stories, participate in giveaways, and more.
Already joined? Log in
LOG IN
SIGN UP
Use your social account:
Or sign up with email:
Title
Mr
Mrs
Ms
Miss
Mx
Dr
This email is already registered. Enter a new email, try signing in or retrieve your password
Why are we collecting this information? Gates Notes may send a welcome note or other exclusive Insider mail from time to time. Additionally, some campaigns and content may only be available to users in certain areas. Gates Notes will never share and distribute your information with external parties.
Bill may send you a welcome note or other exclusive Insider mail from time to time. We will never share your information.
Sign up
We will never share or spam your email address. For more information see our Sign Up FAQ. By clicking "Sign Up" you agree to the Gates Notes Terms of Use / Privacy Policy.
Street address
City
postal_town
State Zip code
administrative_area_level_2
Country
Data
Gates Notes Insider Sign Up FAQ

Q. How do I create a Gates Notes account?

A. There are three ways you can create a Gates Notes account:

  • Sign up with Facebook. We’ll never post to your Facebook account without your permission.
  • Sign up with Twitter. We’ll never post to your Twitter account without your permission.
  • Sign up with your email. Enter your email address during sign up. We’ll email you a link for verification.

Q. Will you ever post to my Facebook or Twitter accounts without my permission?

A. No, never.

Q. How do I sign up to receive email communications from my Gates Notes account?

A. In Account Settings, click the toggle switch next to “Send me updates from Bill Gates.”

Q. How will you use the Interests I select in Account Settings?

A. We will use them to choose the Suggested Reads that appear on your profile page.

BACK
Forgot your password?
Enter the email you used to sign up and a reset password link will be sent to you.
This email is already registered. Enter a new email, try signing in or retrieve your password
Reset Password
Reset your password.
Set New Password
Your password has been reset. Please continue to the log in page.
Log in
Get emails from Bill Gates
Send me updates from Bill Gates
You must provide an email
On
Off
Email me comment notifications
On
Off
On-screen comment notifications
On
Off
This email is already registered
Finish
We will never share or spam your email address. For more information see our Sign up FAQ. By clicking "Continue" you agree to the Gates Notes Terms of Use / Privacy Policy.
You're in!
You're in!
Please check your email and click the link provided to verify your account.
Didn't get an email from us? Resend verification email
Upload a profile picture
Choose image to upload
Uploading...
Uh Oh!
The image you are trying to upload is either too big or is an unacceptable format. Please upload a .jpg or .png image that is under 25MB.
Ok
Title
Mr
Mrs
Ms
Miss
Mx
Dr
Cancel
Save
This email is already registered
Cancel
Save
Please verify email address. Click verification link sent to this email address or resend verification email.
Email and notification settings
Send me updates from Bill Gates
You must provide an email
On
Off
Email me comment notifications
On
Off
On-screen comment notifications
On
Off
Select your interests
Saving Lives
Energy Innovation
Improving Education
Alzheimer's
Philanthropy
Book Reviews
About Bill Gates
Finish
Confirm Account Deactivation
Are you sure you want to deactivate your account?
Deactivating your account will unsubscribe you from Gates Notes emails, and will remove your profile and account information from public view on the Gates Notes. Please allow for 24 hours for the deactivation to fully process. You can sign back in at any time to reactivate your account and restore its content.
Deactivate My Acccount
Go Back
Your Gates Notes account has been deactivated.
Come back anytime.
Welcome back
In order to unsubscribe you will need to sign-in to your Gates Notes Insider account
Once signed in just go to your Account Settings page and set your subscription options as desired.
Sign In
Request account deletion
We’re sorry to see you go. Your request may take a few days to process; we want to double check things before hitting the big red button. Requesting an account deletion will permanently remove all of your profile content. If you’ve changed your mind about deleting your account, you can always hit cancel and deactivate instead.
Submit
Cancel
Thank You! Your request has been sent