Pedestrian Crosswalk Lights: What are the Rules?

We all know that when the white human form appears, all may cross the street safely and legally. But when the orange hand appears and is blinking, does this mean take caution and yield to the oncoming cars that have already been granted with a green traffic light? I know that when the orange hand is solid, you don’t dare step out into the road but I am seeing more and more pedestrians walking during the orange blinking hand, therefore traffic that does have the green light has to stop in the middle of an intersection and allow the pedestrian to continue their journey to the other side. If I were to slam on my brakes during a green light to allow a pedestrian who is crossing on a flashing orange hand, I risk getting hit from behind or honked at quite severely as well as that pedestrian yelling at me. Is the pedestrian at fault and considered to be jaywalking?