Untenable situation with boss and his peers

BLUF: I need to satisfy 3 value stream (VS) managers while reporting to one of them.

I run the shared services group (paint, cleaning, and machine) for a large OEM. The structure here is broken down into VS's with my VS being primarily a piece build operation. The other 2 VS's focus more on final assembly prior to going to shipping to our customers.
My group supports all 3 VS's to ensure delivery from one area to the next and is engaged during various phases of product builds.
In a meeting with my manager and the manager of another VS last week, I committed to a 36 hour turn in one of my shops and made it very clear that this would affect all 3 VS's. My manager was in full agreement with this decision and took the opportunity to highlight how he could control demand in our VS and help me hold my peers accountable for their planning.
The reason for this is there is a perception that our VS gets prioritized over others when push comes to shove.
I made sure to share this plan with my peers the net day when we returned to work and to stress it every-time they would try accelerate their units.
This morning my manager told me that a certain part was high priority and needed to be accelerated. When I reminded him of what I had been saying all week he smiled and said "but I am the differentiater here".
He has a history of doing this - changing his tune as soon as he feels he needs to shift focus to ensue that his metrics stay green. I understand why we need to get these parts out, I just worry that he will further exasperate the issue by doing this.
I refuse to blame him when/if anyone comes asking as it is ultimately my decision.
I have thought about asking that my are be removed from reporting directly to any of the VS's because of these perceptions.

Do I just suck it up and perform as directed or?????

Mac