Our zonehosts have multiple NICs, on multiple subnets. This means that they have multiple default routes defined, so non-local packets are passed to those default routers in a round-robin fashion. In the past, this has not been a problem, because these default routers are actually just routers.
However now, I am creating a set of zonehosts that will be straddling a firewall. And like any good firewall, they will drop packets that are coming in on the "wrong" interface. So here's what I had to do to make this work:
Here's the config for this example:
ce0 (192.168.1.10/24) -> fw interface 192.168.1.1
ce1 (192.168.2.10/24) -> fw interface 192.168.2.1
On the global zone, edit /etc/ipf/ipf.conf to add the following rules for each interface
block out quick on ce0 to ce1:192.168.2.1 from 192.168.2.0/24 to any
block out quick on ce1 to ce0:192.168.1.1 from 192.168.1.0/24 to any
Now all the packets are put on their correct interface.
The only remaining question is "how does this deal with IPMP and link failures". That's something for this afternoon's research.