First: Don't worry, be happy. You have a live tree, which is rather better than a dead one.
Second: Mistakes in styling are (to quote an author who quoted his teacher) like a bad haircut. With time, they can be fixed.
Third: Don't rush to judgement. At the first bonsai demo I attended (the one that got me interested in the art, in fact), the presenter indicated a tree he had on display and said, "I've been pondering the removal of this branch here for the past three years." You don't have to make a perfect bonsai in one sitting, or even in one year. Good thing, too, because you won't be able to. Nobody is able to.
Now, to specifics. Based on what I can see, I'd keep that branch for now and work on the upper parts of the tree. Personally, I think that branch could be useful down the line. The odd direction it's growing in isn't an issue because (I would expect) you can wire it and get it into a more suitable position, and you can always prune back parts of it that don't work without cutting off the whole thing.
A more immediate problem, from what I can see in the photos, is the rather thick branch to the left. It's as thick as, if not thicker than, the main line of the trunk at that level. If I were going to remove anything right now, it would probably be that one.
That said, I'm only seeing the tree in 2-D and not from all angles. Before doing anything drastic, I would suggest hooking up with a local bonsai club. Take your tree to a meeting and ask for advice. Someone could even work on it with you right then and there. It's about the best way to learn.
And finally, feel free to disregard anything I say. I'm by no means an expert myself. I've merely gotten to the point where I'm bold enough to express a few opinions.