According to the Help system, you can import paragraph styles from one doc into another, so you could set one document up with the formatting you want and import it into the next, and the next, and so on.

Alternatively, you could define a template with the paragraph styles you want and attach it to each doc in turn.

I haven't tried it in Open Office Writer, but in MS Word it's easy after you've spent ages learning how to do it.