James O'Malley (O'Brien) is a New York City police officer with a penchant for handing out tickets. For Officer O'Malley there are absolutely no gray areas---black is always black. With his overzealous, no-room-for-discussion ways, he hands out tickets left and right. Continually citing people for old-fashioned ordinances that no one even remembers anymore, such as too many bells on a wagon or a storefront awning being the wrong length, Officer O'Malley has become a pain in the neck to the people on his beat. His superior warns him that justice needs to be tempered with mercy, but O'Malley just doesn't get it.
One day, he pulls John Phillips (Humphrey Bogart) over for having a loud muffler on his car. Although Mr. Phillips asks O'Malley to quickly write the ticket because he is on his way to the first job he's had in years, the officer takes his time about things. By the time Phillips arrives at his job, he's late, and the job has been given to someone else. With a wife and lame daughter at home, Phillips is desperate and tries to pawn his gun; however, when the pawnshop owner won't pay the amount of money Phillips wants, Phillips clobbers the man on the head and then robs the store, a crime for which he ends up being sentenced 2-10 years.
Officer O'Malley's superior believes that O'Malley's focusing on minor infractions drove John Phillips to crime...had O'Malley not stopped Phillips for something as petty as a noisy muffler, he wouldn't have missed out on the job and wouldn't have been driven to the breaking point. The police chief believes that O'Malley needs to get a heart and stop harassing people, and in an effort to either make or break the officer, the police captain demotes O'Malley to the position of school-crossing guard. And it just happens to be the school which John Phillips' daughter, Barbara (Sybil Jason), attends.
Barbara, who walks with a limp, is a kind and loving child, and she soon has stolen the heart of the gruff crossing guard. She tells him her father is working in Canada, but when Officer O'Malley escorts Barbara home one day, he sees a photo of her father and realizes that he is none other than John Phillips. At this point, Officer O'Malley is a changed man, and he seeks to undo the damage he caused the little girl's family. How everything works out is the balance of the film.