When you are hesitant to start working in a new sketchbook, turn to the first page and make a mark...whatever...but make a mark. Draw something that you're familiar with or is easy for you to draw. Doodle drawings are non threatening and fun. You can do these without even thinking about them.
I taught middle school and high school art. The first thing I would have them do in their sketchbooks was to do a creative name design. They'd write their name in large decorative letters to fill the page...then fill the letters and page with patterns, designs, and colors. They loved it. I did, too. I wish I had an example of one of these designs.
This "inside right cover" image was difficult to scan so it looks right...but this is something else I've done and had my students do. I glue a sheet of black paper to the page on the right side...but only apply the glue close to the binding...like I was adding in another page. I either tear out a hole or cut a flap that can be opened to a drawing underneath. This drawing can be something small and easy for you to draw. I then went back and used gold pens and gold spay paint to decorate the black. This sure makes a "grand entrance" to a new sketchbook...and then you're excited to keep "filling your book".
Something else I like to do to my sketchbooks is to paint the covers with abstract designs using brightly colored acrylic paints...so much fun...and what a beautiful book it makes.
I used to feel the same way about starting a painting on a canvas. I'd set it up, then look at it "forever"...afraid I'd mess it up. I'd want to paint...but just couldn't get started...so I'd keep looking at that blank canvas. I began to realize that once I got started...I got so involved I couldn't quit...or didn't want to. So...as I walked by that canvas...I'd grab a brush, pick up some paint or some medium...and smear it on randomly. Then, it was easy to keep painting.
Once you've started your sketches and drawings...if you "feel" like you've messed it up...put it aside for a few days...then look at it again. Things always look better the next day or a few days later. But...if you're absolutely SURE you've messed it up and you feel like it's going to inhibit you from doing future drawings...then collage something over it. Your art is supposed to be fun, enjoyable, and satisfy that "creative urge" inside you.