Configured VPN using BGP to cloud provider at site A and everything worked as expected, then configure VPN using BGP to cloud provider at site B and traffic from site A to site B broke.
Looking at logs, saw traffic from site A reaching site B then getting routed by BGP through the cloud provider back to site A because BGP is advertising that route as the shortest path.
for logs, look at your BGP summary and the BGP routing table to see where subnets are being routed to
site A subnets are on the left and the VPN subnets are on the right in example below;
get router info routing-table
allpath=router, objname=info, tablename=(null), size=0Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
B 10.11.7.1/24 [20/100] via 18.104.22.168, vpn-cloudprovider-1, 00:01:11
B 10.11.8.0/26 [20/100] via 22.214.171.124, vpn-cloudprovider-1, 00:01:11
B 10.12.1.0/28 [20/100] via 126.96.36.199, vpn-cloudprovider-1, 00:01:11
Create a BGP community, then define a no export rule to prevent routes being advertised to cloud provider do not get propagated to external networks. Assign the new rule to the BGP neighbors and perform this task on both site A and B, this will solve the routing issue and all your traffic will flow as designed.
Here is an example of what i did on site A;
config router community-list
set action permit
set match "no-export"
Create route-map that matches the community
config router route-map
edit "outbound to cloudprovider"
set match-community "no-export"
Apply route-map to BGP cloudprovider neighbor
config router bgp
set route-map-out "outbound to cloudprovider"
Clear bgp router to take new configs
exec router clear bgp all