It was annoying but more than that, it's wasting water for no good reason and I knew that it would get worse and more frequently if I don't fix the flush kit that is not sealing the water tank properly.
As I looked online for available options, I noticed that dual flush kit has become very popular. The dual flush kit is basically a flush level mechanism that has two flush settings for your different bathroom needs. In other words, you don't need as much water to flush your #1 compared to your #2
Although there were mixed reviews about this kit, I decided to give it a try on my toilet because according to the specification, it can save you up to $120 per year in water bill.
So I picked it up at Lowe's for $30
For this model, you would flush the level down as you would normally do for your #2 but for your #1, you would pull the level up, which would use only the half of the water compared to the level down.
The instruction was pretty simple and it came with everything you need to replace the existing kit except for the overflow tube (see diagram below).
Basically, it came with everything in the diagram above except for parts highlighted in yellow.
I believe you will need to remove the toilet tank in order to replace the overflow tube and that will be a much bigger job since there's a risk that the new overflow tube may not fit correctly and water could leak outside of tank or water can even leak through the screws that is mounting the two toilet pieces, not to mention the toilet tank can be pretty heavy to life and remove.
Anyways, the instructions were very straightforward and pictures so I was able to replace it in 15 minutes or so and test out the new dual flush mechanism. It also has few settings to adjust the volume of water for #1 vs #2 as well as how much water should be filled in the tank before stopping it.
Overall I'm happy with it and more than anything, I don't hear the water running anymore so it got the job done. Saving some money on the water bill is just added bonus
BTW, the instruction also notes about the toilets that are not compatible with this kit. I wish they have this information outside the box so people will know about it before the purchase... anyways, here's the picture of instruction sheet that describes the ones that are not compatible.
My toilet had an angled flush valve type but the kit came with a part that can fit with angled flush valve type so it was not a problem at all. Let me know if you have any questions!