Select Python comment (#) [solved]

[SpongeB0B]

Hi,

I'm trying to select the Python comment (one line)

#comment example

I'm trying with the so call "Regular Expression" RegEx But so far I only managed to catch only the multi line """ or '''

Bur the one-liner # that seem easier , is not ! I've dig the WWW , but the majority of the example contains bugs !!

It's so weird as for example the coloration done by Vscodium can catch where is the comment (and the Python parser too)



Any ideas or where to look ?
BTW other solution that RegEx are welcome too.

Thanks.

[bowlofred]

Hi,

I'm trying to select the Python comment (one line)

#comment example

I'm trying with the so call "Regular Expression" RegEx But so far I only managed to catch only the multi line """ or '''

Can you show what expression you've tried? Giving an example with what output you expect and what output you're getting instead would help.

[Pedroski55]

Something like this?

import regex

source_file = '/home/peterr/PVE/pandas/flatten_a_dictionary.py'

with open(source_file) as infile:
    text = infile.readlines()

# no need for regex but won't find indented comments
comments = [t for t in text if t[0] == '#']

with open(source_file) as infile:
    s = infile.read()

# .* finds everything in a line except the end of line
# maybe there is some space before the #
# ^\n = not newline, otherwise we pick up the newlines as space before #
f = regex.compile(r'[^\n\s+]?#.*')
res = regex.findall(f, s )

f finds these comments:

Output:
['# make df from lists', '# make df from dictionary', '# problem is info is a list of dictionaries', '# to flatten this data, we need to get info and open it up', '# not nice', '# same as pd.DataFrame(data)', '# make a new dict with lists of ages, genders and locations as well as id and names', '# cunning indented comment here', '# cunning indented comment here', '# successfully flattened                ']

You could probably modify:

comments = [t for t in text if t[0] == '#']

to find indented comments without using a regex. Try it!

[deanhystad]

Maybe the ast (abstract system tree) is a better choice for finding comments in python code.

[SpongeB0B]

Thank you all for you inputs.

I could managed it with tokenize

Work well, but oh god ! the documentation is not at all complete !

[snippsat]

Something like this using tokenize and regex.

import io
import re
import tokenize

def find_python_comments(source: str, CODING_RE: re.Pattern[str]):
    comments = []
    reader = io.StringIO(source).readline
    for tok in tokenize.generate_tokens(reader):
        if tok.type == tokenize.COMMENT:
            line_no = tok.start[0]
            text = tok.string
            # Skip shebang and coding cookie if you don't want them
            if line_no == 1 and text.startswith("#!"):
                continue
            if line_no <= 2 and CODING_RE.fullmatch(text.strip()):
                continue
            comments.append((line_no, text))   # (line number, comment text)
    return comments

# Example
code = """\
#! /usr/bin/env python3
x = 1  # inline comment
s = '# not a comment'
\"\"\"a triple-quoted string with # inside\"\"\"
'''# Hello World '''
# real comment
python.code('# This is not comment') # A comment #Part of comment
"""

if __name__ == '__main__':
    CODING_RE = re.compile(r"#.*coding[:=]\s*[-\w.]+", re.I)
    print(find_python_comments(code, CODING_RE))

Output:
[(2, '# inline comment'), (6, '# real comment'), (7, '# A comment #Part of comment')]

[Pedroski55]

Amazing what one can think up a 4am with the wind howling outside!

source_file = '/home/peterr/PVE/pandas/flatten_a_dictionary.py'
 
with open(source_file) as infile:
    text = infile.readlines()

comments = [t.split('#')[1] for t in text if '#' in t]

If there are more than 1 # in a line:

# if there are more than 1 # in a line then need some more magic, but works with only 1 # per line too
comments = ['_|_'.join(t.split('#')[1:]) for t in text if '#' in t]

[tester_V]

to Pedroski55: I like how you think! You are something...