This is neat! I think yoke does something similar. It was designed for zero-copy deserialization but I think it could be used for the same thing. It seems like it doesn't use Pin because instead of Deref it has a get() method which returns a local lifetime, but I'm not too sure of the specifics.
yep! it's a very similar design. looks like it uses a StableDeref trait from a non-standard-library crate to be safe. That crate was originally released before Pin was established, seems to provide some of the same invariants.
If i weren't so insistent on only using the standard library & understanding everything myself, i probably should have just used yoke instead :P
As written, PinRef::project is unsound: minimal repro.
The issue here is that impl for<'a> FnOnce(&'a T) -> &'a U does not actually mean "for all lifetimes 'a" -- this would be invalid, because T might not be valid for all lifetimes 'a. Instead, it means "for all lifetimes 'a that don't outlive T". Thus if you have two lifetimes 'x and 'y such that 'y outlives 'x, and you have a PinRef<_, &'x _>, you can project with a function that returns a reference with lifetime 'y, since 'y outlives all lifetimes 'a that don't outlive 'x.
I believe this particular exploit can be patched by requiring that T: 'static, though it's not immediately clear to me that that makes the API sound.
I think it's worth particularly questioning this logic:
How we should interpret this is: If we can go from a &T to a &U for an arbitrary lifetime 'a, that means *U is a fixed offset from *T.
Since it's valid for the function to return some 'static reference not derived from the &T, this is not true as written. Of course, a 'static reference does not cause an issue for the desired use-case, but I think it highlights that this isn't the correct line of reasoning.
Instead, I might consider the following reasoning:
Because the function must work for an arbitrary lifetime 'a, we can decide arbitrarily how project determines the lifetime it supplies to the function. In particular, we can choose "whatever the lifetime of valends up being" (looking into the future).
Combined with ensuring that the bound actually is "for any lifetime 'a", this reasoning seems sound to me?
Thanks for the repro!! I knew I was missing something...
Intuitively, it feels like going from small lifetime 'x to larger lifetime 'y isn't a problem on its own. Like, it's not unsound (tho it is weird) to return a 'static reference in the projection.
Instead, I think the problem is that the Deref implementation doesn't obey the lifetime bound of T. I'm not actually sure it's possible to write the trait to make it obey that as-is (might need a to add a phantom field), but T: 'static should close that gap the same.
I like your reasoning too, better phrased, mind if I credit you in an update?
Super neat! Playing around with this a bit, I'm not sure you actually need the Pin. The pointee of an Arc should already be addr stable. If we take a look at the Arc::as_ptr doc it says:
The counts are not affected in any way and the Arc is not consumed. The pointer is valid for as long as there are strong counts in the Arc.
Oh neat I didn't realize that was guaranteed! nice
Originally I wrote things to work for any Deref-ing pointer, not just Arc, before realizing that just makes things way too complex, this further simplification is nice.
fractalbeauty | 6 hours ago
This is neat! I think yoke does something similar. It was designed for zero-copy deserialization but I think it could be used for the same thing. It seems like it doesn't use
Pinbecause instead ofDerefit has aget()method which returns a local lifetime, but I'm not too sure of the specifics.[OP] polywolf | 5 hours ago
yep! it's a very similar design. looks like it uses a
StableDereftrait from a non-standard-library crate to be safe. That crate was originally released beforePinwas established, seems to provide some of the same invariants.If i weren't so insistent on only using the standard library & understanding everything myself, i probably should have just used
yokeinstead :Psnej | 2 hours ago
Nice! yoke will be very useful in code I’m writing; thanks for the reference.
T6 | 2 hours ago
As written,
PinRef::projectis unsound: minimal repro.The issue here is that
impl for<'a> FnOnce(&'a T) -> &'a Udoes not actually mean "for all lifetimes'a" -- this would be invalid, becauseTmight not be valid for all lifetimes'a. Instead, it means "for all lifetimes'athat don't outliveT". Thus if you have two lifetimes'xand'ysuch that'youtlives'x, and you have aPinRef<_, &'x _>, you can project with a function that returns a reference with lifetime'y, since'youtlives all lifetimes'athat don't outlive'x.I believe this particular exploit can be patched by requiring that
T: 'static, though it's not immediately clear to me that that makes the API sound.I think it's worth particularly questioning this logic:
Since it's valid for the function to return some
'staticreference not derived from the&T, this is not true as written. Of course, a'staticreference does not cause an issue for the desired use-case, but I think it highlights that this isn't the correct line of reasoning.Instead, I might consider the following reasoning:
Combined with ensuring that the bound actually is "for any lifetime
'a", this reasoning seems sound to me?[OP] polywolf | 53 minutes ago
Thanks for the repro!! I knew I was missing something...
Intuitively, it feels like going from small lifetime
'xto larger lifetime'yisn't a problem on its own. Like, it's not unsound (tho it is weird) to return a'staticreference in the projection.Instead, I think the problem is that the
Derefimplementation doesn't obey the lifetime bound ofT. I'm not actually sure it's possible to write the trait to make it obey that as-is (might need a to add a phantom field), butT: 'staticshould close that gap the same.I like your reasoning too, better phrased, mind if I credit you in an update?
dov | 49 minutes ago
Super neat! Playing around with this a bit, I'm not sure you actually need the Pin. The pointee of an Arc should already be addr stable. If we take a look at the Arc::as_ptr doc it says:
It is completely possible I am missing some other unsoundness this introduces but here is a version with just Arc https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=6f430a92bbb2409d096c8c7d96532514
[OP] polywolf | 42 minutes ago
Oh neat I didn't realize that was guaranteed! nice
Originally I wrote things to work for any
Deref-ing pointer, not justArc, before realizing that just makes things way too complex, this further simplification is nice.dov | 33 minutes ago
Yeah that tracks. If the goal is to have this Just Work :tm: with Deref-ing pointer than Pin or StableDeref is probably the way to go.
addison | 5 hours ago
Soon we shall haveArc::mapand the nightmare will be overOh no this doesn't do what I thought, I was thinking of mappable-rc.