i have all kinds of strange worlds running around in my skull, so when i get a weird what if idea i just toss it into the world that fits it best and see where it goes.

Your working on a novel, and i imagine you haven't already sold it so your not really working on a deadline. so world building should more than likely come first. but it really depends on how the little story building midgets in your skull like to work.

I try to create a rough world and flesh it out as I go, only really detailing those bits I need for the current plot.

Try plotting your story first, bullet point it or write a paragraph that details each scene, then world build what you need to put some meat on it.

I think most people start with one thing that's easy and rewarding for them to work on until they have a substantial body of work done. For me its dialog. I'm constantly going back and replacing dialog because there's so much of it is blows a fuse in the reader's brain.

I think this is a common kind of problem in the manuscripts I've critiqued; they're strong in one area (plot, or worldbuilding or characterization) but weak in others. The one thing I'd say is if you're not a plotter by nature, you want to grab that bull by the horns before you've written a novel full of whatever is easy for you but still have no story. Plot is *very* hard to retrofit to some arbitrary collection of 100K words. It may be the hardest thing of all to retrofit.

If it helps, David Farland puts it like this.

Setting (world building) first, because character grows out of the setting. What your character knows, his opinions, his job, his education, his experiences, who he knows, likes, and is related to all come out of the setting.

Then the character because the conflict grows out of the character.

My advice: take an existing world (real or fantasy, doesn't matter) from someone else and then try tweaking the detail. You should not use this world for writing because you can easily make things similar which readers might recognize. This world is to practice what works and what would you change in an already existing setting. When you do work on your own world, use the experience you got.

I usually start with an idea and think of what kind of person would do this and/or what kind of world could this be happen in.

By the time I start writing, I have a rough outline of the plot and a vague idea of characters and setting, and I develop those as I write.

Nothing works for anybody. What I do won't work for you, and what works for you won't work for me. there is usually two methods for every author in existence. they work in certain situations and not in others.

I am not one to plan out the background in detail before I start writing. I do have two plotting systems. One is to blast out a rough story to see how the plotting will work out. this is my main method.
The other which I don't do often any more is to write each scene as a short sentence. usually not even complete sentences. When I write these, I will either edit them into full scenes, or I would start writing, trying to hit those scenes in my work.

Usually, though I tend to start with a scene or situation, and write something out fast so I have something that resembles a story. If I have a beginning, middle and end to it, then I am on the right track.
I then go back and build more detail into the world that fits the situation I created. Once I have developed some of the details, I correct the story to fit the details I worked out to make the story work.

This does not always work for me, though, as the story may change dramatically.
In my Waxy dragon story series i did, they were blasted out and the rules developed over the sixty some stories. when I went back to actually write the stories, the rules changed, the concept changed, the stories changed. Not one of the stories I have written so far will have any resemblance to what I wrote before, other than the basic plot and even then it will change dramatically.

I have one novel that takes place on another planet and a fantasy novel-closer to classical fantasy compared to the Urban Fantasy I'm doing now-that is mostly done. In both I formed some ideas of the planet before I started worked but added details as I wrote the story. I knew the planet had less land mass than Earth before I started but designed a poisonous flying snake as I needed something to bite one of the MCs.

You could make careful written-down notes...but you run the risk of obsessing over them, to the point where you don't write your story.

I think the best thing to do before you start working on a novel is figure out the rules of the world. I'm not saying you need to know what color flag the kingdom has flying, but general rules.

FOR EXAMPLE,
How is power delegated?
If there's magic, how exactly does the magic work, and affect the user?
Status of characters, level of technology, family dynamics, etc.

The rules.

I think once you know the rules, then you can figure out the details as you go along.
~Sheena