Bitcoin Q&A: SegWit and transaction malleability

“Can you explain transaction malleability?”
“I thought that the transaction ID cannot be changed,
even in a very small way. That is a great question, Sally.
The transaction ID cannot be changed
because it represents a hash of the transaction.
That is the theory. Here is the practice.
The transaction ID is a hash of every part
of the transaction, except for the signatures.
In the past, the transaction ID included signatures.
That was a problem, because the only part…
of the transaction not signed is the signature itself.
As a result, someone could change the signature
in such a way that it affects the transaction ID.
So I [could] produce an identical copy
of a transaction before it is confirmed,
being propagated to [the network].
I [can] then modify the signature. Since I don’t
have the private key, I can’t change the signature…
to sign something else.
I can’t change the inputs or the outputs because
they are signed. I can’t forge the signature.
But I can add something to the signature
that doesn’t change its function.
Keep in mind, the signature is a number.
Think of the signature as the number ‘5.’
It is not, but let’s use that as
a simple arithmetic example.
If I change how it is encoded in the transaction
and write ’05.’ Arithmetically, it is the save as ‘5.’
But when written inside a transaction and serialized in
binary, that zero changes the data in the transaction ID,
which will produce a different transaction ID.
If you validate that signature arithmetically,
’05’ is the same as ‘5.’ It doesn’t really matter.
It doesn’t change the outcome, like re-writing
the signature as ‘5+1-1.’ [Arithmetic variations].
But that is not exactly how they work because we
are dealing with binary, a strictly serialize format.
The developers kept finding ways that you could
modify a signature without changing its actual value;
modify how it was written, not what number it was.
Therefore, when someone validated it,
‘5’ was a valid signature for that transaction.
But because it was written differently,
when you calculate the transaction ID,
this new copy has a different transaction ID.
You have produced a second transaction that signs
the same amount, to the same destination address,
and everything is the same, except
the transaction ID has changed.
That can cause a lot of problems. If any
other transactions referred to the original,
but [a slightly modified] copy is
confirmed [instead], it breaks that chain.
This affects things like the Lightning Network.
How is this problem fixed? Very simply.
You can take the signature out of the transaction ID.
The transaction ID is not calculated with the signatures.
In a SegWit transaction, the place
where the signatures used to be is empty.
It has a specific fixed format in
[a separate witness data structure].
As a result, you can’t modify it.
The signature has been segregated.
Because it is outside, you can’t modify it
in order to modify the transaction ID.
If you try to modify it, you will modify the part of the
transaction which is signed and therefore invalidate it.
“Can you please clarify why SegWit makes
it easier for the Lightning Network to run?”
“Is it always the case that two parties need to reserve
the same amount of funds for a payment channel?”
Let’s break that into two parts.
Why does SegWit make Lightning work?
The simple answer is, the fundamental feature
introduced by SegWit was to fix transaction malleability.
It makes it impossible for a third-party to
malleate a transaction to produce a different ID,
for the same spendable amount.
This is important because, in the case of Lightning,
parties are exchanging signed bitcoin transactions…
called commitments, which are not
broadcast to [the first layer Bitcoin] network.
They depend on [being able to]
build transactions upon each other.
You need to make sure that the transaction ID
of the previous transaction cannot be changed,
even [when] it hasn’t been confirmed [yet].
You cannot have transaction malleability. It would make
the Lightning Network more difficult to implement.
This is why we don’t see Lightning implemented on
Bitcoin forks that have not fixed transaction malleability.
[Fixing] transaction malleability was
a primary reason for [adding SegWit].
Once you have fixed transaction malleability,
you can chain transactions together in a series,
knowing that you can’t break that chain due to
problems in the validation [of those transactions].
The second question:
“Is it always the case that two parties need to reserve
the same amount of funds for a payment channel?”
No. In fact, you can’t even do that today [yet].
You would need the ability to create…
the channel funding transaction jointly,
through a multi-party transaction.
That feature is expected to be introduced in the BOLT
specification 1.1, coming towards the end of 2019.
Today, when a payment channel is opened, it is created,
funded, and signed by one party asymmetrically.
There is balance only on one side, belonging
to the party that initiated the payment channel.
If I open a channel to someone else, the balance is on
my side; the other party has [nothing] on their side.
If we want to have capacity on the other end,
they must open a channel to me.
But ideally, we would both commit the same amount.
You put half on your end, I put half on my end.
That would make a channel
that is balanced on both sides.
That’s not how it works today,
but it is planned for the future.
It will make balancing channels easier. For now,
channels are one-sided with the starting capacity.
Once you make a few payments,
the balance will move to the other side.
From a practical perspective, when you start a new node,
all of the channels are funded on your side,
so you must wait or ask other nodes to open channels
back to you, so you can start receiving payments.
You cannot receive on Lightning until someone
opens a channel to you, with balance on their side.
“As I understand it, SegWit transactions are seen
as ‘anyone-can-spend’ by nodes that are not updated.”
“Is this a problem?” It is a problem for those
nodes, because they are making assumptions…
about what is [spendable] on the blockchain,
but it is not a problem for the rest of the network.
You might try to configure those nodes to spend these
transactions, operating under the assumption that…
they are spendable, which is incorrect.
The consensus rules in effect on the network
today, being enforced by every node and miner…
with few exceptions, are that those transactions
are not spendable by anyone [except the signer].
They are spendable by a Segregated Witness signature.
If you tried to produce a transaction that
spends these coins and transmit it to the network,
anyone following the current rules would not only
reject that transaction and refuse to propagate it,
but they will also disconnect from your node for lying.
Everyone you send it to will ban your node. You will be
cut off from the network for trying to propagate lies.
The idea that SegWit transactions are spendable by
anyone, under the [current] consensus rules, is a lie.
That lie will not be propagated. A node that pretends
Satoshi’s coins from the first blocks are spendable…
by anyone, while presenting
a fake signature from Satoshi,
is no different.
[They are both] invalid. You can [try] to violate the rules
by pretending that SegWit transactions are spendable…
by anyone, but they are not according to
the current rules. They require a witness.
[You can] pretend that Satoshi’s original coins from the
first block are spendable by you, without the signature,
but they are not according to the consensus rules.
There is no such thing as half violating the rules.
You can configure your node with any rules you want,
including that Satoshi’s coins are spendable without
a signature, just as you can leave your node with…
an incorrect configuration to believe that
SegWit transactions are anyone-can-spend.
They are not. If you try to propagate these lies against
the rules of consensus, the network will ban you.
Your transaction won’t go anywhere.
A lot of people claimed that storing funds with SegWit
carried additional risk, and that was proven false.
Resoundingly, by the fact that there are
now plenty of coins stored with SegWit.
I can tell you that all of my cold storage savings
are inside multi-signature SegWit addresses.
If anybody wants to pretend those are anyone-can-
spend, be my guest. See how the network reacts.
I am quite comfortable putting my money where
my mouth is. I think that is all I need to say about that.
“SegWit improve scalability, but is it
enough for the future of mass adoption?”
“Is scalability not one of the main issues
with cryptocurrency? How can it be solved?”
SegWit improves scalability, but it is
not enough for future mass adoption.
We cannot solve scalability permanently.
There is no such thing as a final solution.
Scalability is solved differently at different [milestones].
You need a scalability solution for each [progression].
The problems we need to solve now, for this scale,
are different from the problems we will need to solve…
for the next scale, and you never solve them completely.
We saw this on the internet. The problems we
solved in order to scale for email were different…
from the problems for scaling attachments and images,
which were different than those for quality voice,
which are different from the problems in scaling
the internet for 4K HD Ultra quality videos on Netflix.
At each level, when you solve one scaling problem,
you are opening the door for new applications.
Now you must solve a different scaling problem.
There is no end [scaling solution].
Also, if you prematurely optimize and
scale for problems that don’t yet exist,
you will shift the problem somewhere else.
In the case of cryptocurrencies, [often] damaging
decentralization to solve a problem you don’t have yet.
For example, if we introduced 32 megabyte blocks
today as a scalability solution for the future,
for transactions that don’t exist yet, we end up
with a blockchain that is mostly empty,
makes denial-of-service attack easier, shifts
a lot of power to miners, and centralizes nodes.
Damaging decentralization.
You don’t want to solve scaling prematurely.
Can you solve it? Yes. SegWit is just the first step.
There will be dozens and dozens of optimizations.
Likely, the next step is Schnorr signatures, which
will introduce a 25-30% improvement in capacity.
It builds upon SegWit.
Then signature aggregation, which will not be in the first
iteration, but can be implemented in a future iteration.
That could be a massive increase in capacity, because
it allows you to have one signature per transaction…
instead of one signature per input.
Potentially in the future, we could even have one
signature per block for all aggregated transactions.
Scalability is solved again and again
with different optimizations. In my opinion,
some of these optimizations will probably
involve increasing the base block size,
but we don’t need to start with that.
“Is scalability one of the main issues of
cryptocurrencies? How can it be solved?”
It is not one of the main issues.
It depends on what you are trying to achieve.
Right now, Bitcoin has enough capacity to handle
all of the transactions that people need to send,
at reasonable fees, without any problems.
It did as well when it wasn’t being jammed by miners
with fake transactions in order to drive fees up.
We did have some aberrations in fees, but this
was mostly deliberate spamming of the network.
In the future, we will see problems with fees and
capacity crunches again, which will force people to…
design smarter fee management
[algorithms] within wallets,
and use techniques like replace-by-fee
(RBF) and child-pays-for-parent (CPFP).
People who are willing to pay [more] for
transactions can signal that intent to the miners.
We will have a fee market that is somewhat healthier
than just shifting the [scaling] costs to full nodes.
A lot of the problems with scalability can be solved
by building second-layer networks, like Lightning.
Even above and beyond that, third-layer networks.
In my opinion, the problems we need to solve
in the first layer are more on the privacy side.
“Hi Andreas. At the end of the December Q&A [session],
you said that we have a limited window of time left…
to make Bitcoin completely anonymous,
and second-layer [solutions won’t be enough].”
“You also said that 100% anonymity will
become table stakes in all cryptocurrencies.”
“Do you think Bitcoin will get anonymity
in time before that window closes?
I have been talking about this idea
since 2013, that protocols ossify.
‘Ossify’ from the Latin word “os”, which means bone.
Basically, they become harder and harder [to change].
What does that mean? When you have a protocol like
Bitcoin, or as we have seen with others like TCP/IP,
which becomes successful and is broadly used,
people build software and hardware around it.
They start embedding the software into various devices.
For example, we already have a number
of companies making node software.
Every time something changes with the
protocol, these nodes need to be updated.
Over time, the Bitcoin protocol will be embedded deeper
and deeper in hardware, in the firmware of chips.
In that case, it becomes more difficult to update.
Think about your Wi-Fi router, for example.
Its implementation of TCP/IP is embedded in firmware.
To update it, you will need to load the new firmware…
and reboot your router.
There are so many devices around you
that have embedded protocols in them.
The idea here is, the longer the protocol survives,
the more mainstream and useful it becomes.
it will gradually become more difficult to change.
That means the window of opportunity for making
changes to the core protocol, the base layer of Bitcoin,
is beginning to close.
It will become more difficult to make changes,
especially the controversial changes.
I think privacy [enhancements]
will be a controversial change.
Consider that changes to [add] privacy will
affect many organizations which currently…
have obligations under the law, like Know-Your-Customer
and anti-money laundering reporting regulations.
Can they support a Bitcoin protocol with strong privacy
protections in it, where they can’t know everyone…
who is participating in a transaction, [and]
they have even more limited access [to data]?
I expect those companies will resist
the introduction of privacy features.
We may see difficulty in introducing these
features [by any other method] than a soft fork.
Fortunately, I think that most of the interesting
privacy features can be introduced by a soft fork.
The protocol can still be extended by soft forks
thanks to a very important change in 2017.
That was the introduction of SegWit.
One of the critical features was versioning.
With SegWit versioning, the scripting language
used within a transaction now has a version number.
That version number can introduce new
functionality without doing a hard fork.
The first version of SegWit introduced in 2017 was
version 0, because we count things from zero.
It is actually in the protocol as the
prefix of a SegWit script, the number zero.
We can upgrade that by adding new versions.
The proposed changes for SegWit version 1…
include some important new privacy techniques:
Schnorr signatures, Taproot, Graftroot, and
Merkilized Abstract Syntax Trees (MAST).
I expect we will see those introduced in Bitcoin
before the end of 2020, as a soft fork.
I am hoping that is not the end, that we will
see other changes that further enhance privacy.
Fortunately, what works to the benefit of Bitcoin
is the introduction of other cryptocurrencies…
that are privacy-focused, which allows
for extensive research to be done.
For example, we have discovered some potential bugs.
I will talk about that in one of the next questions.
The zk-SNARK implementation
in Zcash is quite a big challenge.
But everybody in the ecosystem
learns from mistakes and bugs like that.
I expect we will see more privacy features.
I don’t think we will have problems introducing them
in Bitcoin, but we will see. This is a big challenge ahead.

12 thoughts on “Bitcoin Q&A: SegWit and transaction malleability

  1. Hi Andreas Love your Explanations but the Sound is So Horrible what a Pity !! As we Say Shoemakers are Always the Least Well Shod

  2. I loved the video but the sound is terrible. Please get a good microphone some people who may not know you may find the video 'amateurish' and not care to keep watching, and we want them to keep watching!

  3. That explanation of why signatures were removed from transaction ids doesnt make sense to me. Why couldnt limits be put on the signature so that it could be included in the transaction. Like if it is 5 then you have to write it 05. Always 2 digits. Idk that smells fishy. Is that the reason why bch broke off

  4. I don't think bitcoin will ossify too much. Look at Android: it's way more complicated than bitcoin (its implementation weighs 1+ GB, while Bitcoin Core only weighs 70 MB), it is bound to hardware, and still people manage to upgrade Android every year. Currently the most popular Android is Marshmallow, which was released in 2015, so there is a few years worth of delay, yet people do upgrade eventually. Ubuntu is still more complicated than Android (the distribution weights 4+ GB), yet people somehow manage to upgrade every two years. Even dollar hasn't ossified yet, as there are still new kinds of financial instruments denominated in dollar being introduced. Still, bitcoin may run over the old implementation of TCP/IP, but it doesn't have to be some old implementation of bitcoin.

  5. Wasabi wallet is just the beggining, there is a broad demand for privacy, and most bitcoiners are maximalists, we dont risk being exposed to scams.

Leave a Reply

Your email address will not be published. Required fields are marked *