您的位置:新葡亰496net > 奥门新萄京娱乐场 > 新葡亰496net命令系统,粗解之命令行执行

新葡亰496net命令系统,粗解之命令行执行

发布时间:2019-12-10 13:47编辑:奥门新萄京娱乐场浏览(66)

    前言

    摘要: 基于Django version 1.11.0 . alpha版本

    首先要弄明白,manage.py运行的每一个命令都是去运行django/core/management/commands/ 目录下以这个命令命名的.py文件,这个文件中有一个继承basecommand的command类,最后这命令运行的其实就是command类的handle函数。我们如果自己想添加一个command,那么也只要在即的project中创建一个management/commands/包, 包下面是我们的以命令 .py够成的文件,这个文件中有一个继承basecommand的command类,然后有一个实现自己的处理逻辑的handle函数就好了。

    G:python3_djangoDFpromypro (win)(py3_django) λ python manage.py celery worker --loglevel=infoTraceback (most recent call last):  File "manage.py", line 10, inexecute_from_command_line(sys.argv)  File "C:virtualenvpy3_djangolibsite-packagesdjangocoremanagement__init__.py", line 364, in execute_from_command_line    utility.execute()  File "C:virtualenvpy3_djangolibsite-packagesdjangocoremanagement__init__.py", line 356, in execute    self.fetch_command(subcommand).run_from_argv(self.argv)  File "C:virtualenvpy3_djangolibsite-packagesdjangocoremanagement__init__.py", line 206, in fetch_command    klass = load_command_class(app_name, subcommand)  File "C:virtualenvpy3_djangolibsite-packagesdjangocoremanagement__init__.py", line 40, in load_command_class    module = import_module('%s.management.commands.%s' % (app_name, name))  File "C:virtualenvpy3_djangolibimportlib__init__.py", line 126, in import_module    return _bootstrap._gcd_import(name[level:], package, level)  File "", line 978, in _gcd_import  File "", line 961, in _find_and_load  File "", line 950, in _find_and_load_unlocked  File "", line 655, in _load_unlocked  File "", line 678, in exec_module  File "", line 205, in _call_with_frames_removed  File "C:virtualenvpy3_djangolibsite-packagesdjcelerymanagementcommandscelery.py", line 6, infrom djcelery.management.base import CeleryCommand  File "C:virtualenvpy3_djangolibsite-packagesdjcelerymanagementbase.py", line 59, inclass CeleryCommand(BaseCommand):

     

    鉴于笔者水平有限,文中不免出现一些错误,还请多多指教!

    django项目中的manage.py  command,他会执行manage.py中的execute_from_command_line函数

    File "C:virtualenvpy3_djangolibsite-packagesdjcelerymanagementbase.py", line 60, in CeleryCommand

    django的命令行在整个的django web开发中都会经常用到,而且是必须得用到。所以,能够了解下django的命令行实现其实是非常有帮助的。

    新葡亰496net 1

    if __name__ == "__main__":

        os.environ.setdefault("DJANGO_SETTINGS_MODULE", "testdjango.settings")

        """

        中间忽略

        """

        execute_from_command_line(sys.argv)

    options = BaseCommand.option_list

    如果大家比较关心django命令的详细说明和使用,可以查看这里。

    好了,下边是正文....
    首先大概看一下Django 项目的主要目录,初步建立一下Django源码的世界观。

    而关于execute_from_command_line函数

    AttributeError: type object 'BaseCommand' has no attribute 'option_list'

     

    ├── django          //工程代码存放路径
    ├── docs            //文档
    ├── extras          
    ├── js_tests        //测试
    ├── scripts         //脚本
    └── tests           //单元测试
    

    def execute_from_command_line(argv=None):

        """

        A simple method that runs a ManagementUtility.

        """

        utility = ManagementUtility(argv)

        utility.execute()

    解决办法:pip install django==1.8 

    命令行执行入口

    Django核心代码主要在django目录下边

    这里重点是ManagementUtility.execute函数,它主要是对输入的参数命令进行拆分和验证,然后调用ManagementUtility.fetch_command去调用ManagementUtility.load_command_class获取要执行的command类,也就是定义在django/core/management/commands/目录下的命令文件中的command类。

    原因: 

    Celery == 3.1 django == 1.10 django-celery == 3.1.17

    这个搭配运行有问题

    BaseCommand.option_list在Django 1.8中被弃用,并在Django 1.10中删除。看起来Django的扩展已经更新,但从那以后还没有一个新的版本。

     

    django/
    ├── apps(app模块)
    ├── bin(可执行命令)
    ├── conf(配置)
    ├── contrib(其他开发者贡献代码)
    ├── core(核心组件)
    ├── db(ORM模块)
    ├── dispatch
    ├── forms(表单模块)
    ├── http
    ├── middleware(中间件)
    ├── template(模板)
    ├── templatetags
    ├── test(测试代码)
    ├── urls(url路由模块)
    ├── utils(工具代码)
    └── views(视图模块)
    

    然后开始执行self.fetch_command(subcommand).run_from_argv(self.argv),即command(继承自BaseCommand)类的run_from_argv函数,这个函数会去执行目录下的对应命令文件的command类的execute函数,这个函数最终将执行output = self.handle(*args, **options),也就是在command类的handle函数。因此,这里我们就可以在django/core/management/commands包类找到对应的命令文件,去跟踪对应的handle函数,然后去查看对应的handle函数执行就可以了。

    django通过django-admin.py和manage.py来执行命令,以下是这两个文件的源码:

    在django中我们常用的命令主要有两个,一个是django-admin,一个是xxxx,我们先看一下django-admin
    1、命令位置

    现在我们以manage.py runserver为例:

    1 from django.core import management
    2 
    3 if __name__ == "__main__":
    4     management.execute_from_command_line()
    
    lion@localhost:~/django/django$ whereis django-admin
    django-admin: /usr/local/bin/django-admin /usr/local/bin/django-admin.py /usr/local/bin/django-admin.pyc
    

    我们查看django/core/management/commands/runserver.py中的handle函数即可

    它们都调用了management模块下的execute_from_command_line()方法。

    2、命令内容

    def handle(self, *args, **options):

        from django.conf import settings

        if not settings.DEBUG and not settings.ALLOWED_HOSTS:

            raise CommandError('You must set settings.ALLOWED_HOSTS if DEBUG is False.')

        """

        中间做一些配置检查,环境的准备,细节省略

        """

        self.run(**options)

    这个方法是在django/core/management/__init__.py中定义:

    lion@localhost:~/django/django$ cat /usr/local/bin/django-admin
    #!/usr/bin/python
    
    # -*- coding: utf-8 -*-
    import re
    import sys
    
    from django.core.management import execute_from_command_line
    
    if __name__ == '__main__':
        sys.argv[0] = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv[0])
        sys.exit(execute_from_command_line())
    

    这里重点关注run函数,run函数调用inner_run(),在inner_run中重点关注以下代码块:

    1 def execute_from_command_line(argv=None):
    2     """Run a ManagementUtility."""
    3     utility = ManagementUtility(argv)
    4     utility.execute()
    

    其实对比不难发现,django-admin命令其实对应的是django源码中的.django/bin/django-admin.py这个文件。
    django-admin.py 引用了django.core中的management,并调用了其execute_from_command_line函数。
    注:在最新版中django-admin和manage.py中调用的都是execute_from_command_line函数了,较旧版本的django中可能不同。
    所以要分析django的命令系统,就要从execute_from_command_新葡亰496net命令系统,粗解之命令行执行。line函数入手。
    execute_from_command_line函数定义:

    try:

        handler = self.get_handler(*args, **options) #这个handler其实就一个get_uwsgi_application返回的application对象

        run(self.addr, int(self.port), handler,   

        ipv6=self.use_ipv6, threading=threading)

    except socket.error as e:

    实现非常简单:生成一个ManagementUtility对象,并让这个对象执行相应的命令行命令。所以主要的工作都是在ManagementUtility这个类中实现的。

    def execute_from_command_line(argv=None):
        """
        A simple method that runs a ManagementUtility.
        """
        utility = ManagementUtility(argv)
        utility.execute()
    

    首先通过get_handler获取一个处理request请求的uwsgi对象(这个在django正式应用环境中,写在了项目的wsgi.py中的)。然后调用django/core/servers/basehttps.py中的run方法(在runserver.py的顶层中可以看到这样的语句from django.core.servers.basehttp import get_internal_wsgi_application, run)。

     

    函数初始化ManagementUtility类,传入argv(也就是命令行参数)参数,并执行execute方法
    execute方法:

    django/core/servers/basehttps.py的这个run方法首先会去创建一个TCPServer来代替apache或者nginx作为web服务器的作用,去监听端口,(我们知道,web服务器的作用就是监听端口,接收、管理请求数据,并返回相应数据)。apache,ngnix等web服务器是一个在应用层的采用http协议进行数据传输的应用,他的下层还是一个基于tcp的socket server,所以内部服务器使用socket server作为web服务器。源码如下:

    ManagementUtility类

    def execute(self):
        """
        Given the command-line arguments, this figures out which subcommand is
        being run, creates a parser appropriate to that command, and runs it.
        """
        try:
            subcommand = self.argv[1]
        except IndexError:
            subcommand = 'help'  # Display help if no arguments were given.
    
        # Preprocess options to extract --settings and --pythonpath.
        # These options could affect the commands that are available, so they
        # must be processed early.
        parser = CommandParser(None, usage="%(prog)s subcommand [options] [args]", add_help=False)
        parser.add_argument('--settings')
        parser.add_argument('--pythonpath')
        parser.add_argument('args', nargs='*')  # catch-all
        try:
            options, args = parser.parse_known_args(self.argv[2:])
            handle_default_options(options)
        except CommandError:
            pass  # Ignore any option errors at this point.
    
        no_settings_commands = [
            'help', 'version', '--help', '--version', '-h',
            'startapp', 'startproject', 'compilemessages',
        ]
    
        try:
            settings.INSTALLED_APPS
        except ImproperlyConfigured as exc:
            self.settings_exception = exc
            # A handful of built-in management commands work without settings.
            # Load the default settings -- where INSTALLED_APPS is empty.
            if subcommand in no_settings_commands:
                settings.configure()
    
        if settings.configured:
            # Start the auto-reloading dev server even if the code is broken.
            # The hardcoded condition is a code smell but we can't rely on a
            # flag on the command class because we haven't located it yet.
            if subcommand == 'runserver' and '--noreload' not in self.argv:
                try:
                    autoreload.check_errors(django.setup)()
                except Exception:
                    # The exception will be raised later in the child process
                    # started by the autoreloader. Pretend it didn't happen by
                    # loading an empty list of applications.
                    apps.all_models = defaultdict(OrderedDict)
                    apps.app_configs = OrderedDict()
                    apps.apps_ready = apps.models_ready = apps.ready = True
    
            # In all other cases, django.setup() is required to succeed.
            else:
                django.setup()
    
        self.autocomplete()
    
        if subcommand == 'help':
            if '--commands' in args:
                sys.stdout.write(self.main_help_text(commands_only=True)   'n')
            elif len(options.args) < 1:
                sys.stdout.write(self.main_help_text()   'n')
            else:
                self.fetch_command(options.args[0]).print_help(self.prog_name, options.args[0])
        # Special-cases: We want 'django-admin --version' and
        # 'django-admin --help' to work, for backwards compatibility.
        elif subcommand == 'version' or self.argv[1:] == ['--version']:
            sys.stdout.write(django.get_version()   'n')
        elif self.argv[1:] in (['--help'], ['-h']):
            sys.stdout.write(self.main_help_text()   'n')
        else:
            self.fetch_command(subcommand).run_from_argv(self.argv)
    

    def run(addr, port, wsgi_handler, ipv6=False, threading=False):

        server_address = (addr, port)

        if threading:

            httpd_cls = type(str('WSGIServer'), (socketserver.ThreadingMixIn, WSGIServer), {})

        else:

            httpd_cls = WSGIServer

        httpd = httpd_cls(server_address, WSGIRequestHandler, ipv6=ipv6)

        if threading:

            httpd.daemon_threads = True

        httpd.set_app(wsgi_handler)

        httpd.serve_新葡亰496net,forever()

     

    此方法主要解析命令行参数,加载settings配置,如果setting配置成功则执行django.setup函数(此函数主要是加载App),最后一步调用的核心命令为fetch_command命令,并执行run_from_argv函数
    先看一下fetch_command函数

    首先创建一个WSGIServer,关于创建这个WSGIServer需要特别关注一个参数,那就是WSGIRequestHandler,这个WSGIRequestHandler在最后的调用process_response()的时候会用到,而run函数里面的wsgi_handler,其实就是get_uwsgi_application返回的uwsgi对象。这个在最后也会用到。

    python是一门面向的对象的语言,django作为python的一个著名web框架,它所使用当然也是面向对象的思想。所以我们在分析源码的时候应该尽量用面向对象的思想去思考。

    def fetch_command(self, subcommand):
        """
        Tries to fetch the given subcommand, printing a message with the
        appropriate command called from the command line (usually
        "django-admin" or "manage.py") if it can't be found.
        """
        # Get commands outside of try block to prevent swallowing exceptions
        commands = get_commands()
        try:
            app_name = commands[subcommand]
        except KeyError:
            if os.environ.get('DJANGO_SETTINGS_MODULE'):
                # If `subcommand` is missing due to misconfigured settings, the
                # following line will retrigger an ImproperlyConfigured exception
                # (get_commands() swallows the original one) so the user is
                # informed about it.
                settings.INSTALLED_APPS
            else:
                sys.stderr.write("No Django settings specified.n")
            sys.stderr.write(
                "Unknown command: %rnType '%s help' for usage.n"
                % (subcommand, self.prog_name)
            )
            sys.exit(1)
        if isinstance(app_name, BaseCommand):
            # If the command is already loaded, use it directly.
            klass = app_name
        else:
            klass = load_command_class(app_name, subcommand)
        return klass
    

    接下来就到了serve_forever函数,serve_forever调用select模型监听,如果有请求到来,就执行_handle_request_noblock(),真正的执行是

    ManagementUtility具有3个属性,我们可以从它的__init__函数中看到。

    这个fetch_command函数类似一个工厂函数,由get_commands函数扫描出所有的子命令,包括managemen中的子命令和app下的managemen中commands的子命令(自定义),然后根据传入的subcommand初始化Command类。
    如果子命令不在commands字典内的话,会抛出一个“Unknown command”的提示,如果子命令存在则返回初始化的Command类。
    接着视角在返回到execute函数中,接着

    process_request() -> finish_request -> RequestHandlerClass,注意了,这个RequestHandlerClass就是在创建socket server的时候传递的参数WSGIRequestHandler,也就是django/core/servers/basehttps.py中的WSGIRequestHandler类。

     

    self.fetch_command(subcommand).run_from_argv(self.argv)
    

    WSGIRequestHandler会去调用自己的handle函数,为什么会去调用自己的handle函数呢,我们看源码。

    1     def __init__(self, argv=None):
    2         self.argv = argv or sys.argv[:] # 从传入的参数获得,如果没有传入参数就从sys.argv中去取
    3         self.prog_name = os.path.basename(self.argv[0])
    4         if self.prog_name == '__main__.py':
    5             self.prog_name = 'python -m django'
    6         self.settings_exception = None
    

    将会调用fetch_command(subcommand)初始化Command类的run_from_argv方法。run_from_argv由各个Command的基类BaseCommand定义,最终将会调用各个子类实现的handle方法。从而执行子命令的业务逻辑。
    至此,命令调用的逻辑基本完成。


    class BaseRequestHandler:

    """Base class for request handler classes."""

        def __init__(self, request, client_address, server):

        self.request = request

        self.client_address = client_address

        self.server = server

        self.setup()

        try:

             self.handle()

        finally:

            self.finish()

    self.argv:命令行信息,包括命令和参数

    笔者随笔:
    通过阅读这一部分的代码,其中最值得学习的地方在于fetch_commands函数,这是一个运用工厂方法的最佳实践,这样不但最大程度的解耦了代码实现,同时使得命令系统更易于扩展(App 自定义子命令就是一个很好的说明)
    再有一点就是Command基类的定义,对于各种子命令的定义,基类完整的抽象出了command业务的工作逻辑,提供了统一的命令调用接口使得命令系统更易于扩展。

    上面贴出的源码是BaseRequestHandler类,它在__init__函数中调用了handle函数。但是这跟WSGIRequestHandler有什么关系,现在是要WSGIRequestHandler去调用它的handle函数。其实我们从BaseRequestHandler的继承关系中是可以看到,BaseRequestHandler最终继承自BaseRequestHandler类,这个类在它的__init__函数中调用了handle函数,这个我们可以在上面贴出的源码中可以看到。也就是在实例化WSGIRequestHandler的最后会调用handle函数。那关于WSGIRequestHandler类的handle函数,其源码如下:

    self.prog_name:程序名

    def handle(self):

        """Copy of WSGIRequestHandler, but with different ServerHandler"""

        self.raw_requestline = self.rfile.readline(65537)

        """

        中间省略

        """

        handler = ServerHandler(

        self.rfile, self.wfile, self.get_stderr(), self.get_environ()

        )

        handler.request_handler = self      # backpointer for logging

        handler.run(self.server.get_app())

    self.settings_excepiton:settings的异常信息,发现settings的设置有异常,会将异常信息存在这个变量里面

    handle函数里面调用ServerHandler.run,run函数的源码如下:

     

    def run(self, application):

        """Invoke the application"""

        """

        中间忽略

        """

        self.setup_environ()

        self.result = application(self.environ, self.start_response)

    ManagementUtility主要的方法是execute(),它完成了command执行的所有过程。

    run函数的关键就看self.result = application(self.environ, self.start_response),而这里的application就是传进来的wsgi_handler,他是一个具有__call__函数的实例,所以application()就会调用自身的__call__()函数,这个就到了django服务在正式环境中对url请求的处理流程了。详见《django框架在正式环境中的请求处理流程分析www.jianshu.com/writer#/notebooks/14133407/notes/14482608》

    1. 我们知道,django的命令行是具有一定的格式的,都是 command subcommand [arguments],arguments有时是可选的。所以execute方法第一步就是获得subcommand,以便确定后续执行什么任务。

     

    1         try:
    2             subcommand = self.argv[1]
    3         except IndexError:
    4             subcommand = 'help'  # Display help if no arguments were given.    
    

    这里提一下,为什么不先获取command呢?其实command是系统用来找程序入口的。

    1. 用命令解析器CommandParser解析命令行。CommandParser继承了argparse模块的ArgumentParser类,但它只是对ArgumentParser的异常处理进行了加强。

     

     1 class CommandParser(ArgumentParser):
     2     """
     3     Customized ArgumentParser class to improve some error messages and prevent
     4     SystemExit in several occasions, as SystemExit is unacceptable when a
     5     command is called programmatically.
     6     """
     7     def __init__(self, cmd, **kwargs):
     8         self.cmd = cmd
     9         super().__init__(**kwargs)
    10 
    11     def parse_args(self, args=None, namespace=None):
    12         # Catch missing argument for a better error message
    13         if (hasattr(self.cmd, 'missing_args_message') and
    14                 not (args or any(not arg.startswith('-') for arg in args))):
    15             self.error(self.cmd.missing_args_message)
    16         return super().parse_args(args, namespace)
    17 
    18     def error(self, message):
    19         if self.cmd._called_from_command_line:
    20             super().error(message)
    21         else:
    22             raise CommandError("Error: %s" % message)
    

     

    argparse官方文档 | argparse用法总结

    3. 解析器解析出了subcommand的arguments,然后fetch_command根据subcommand导入相应的command包并生成相应的command对象,然后调用command对象的print_help方法或者run_from_argv方法去执行相应的命令。

     1         if subcommand == 'help':
     2             if '--commands' in args:    # only print the commands only
     3                 sys.stdout.write(self.main_help_text(commands_only=True)   'n')
     4             elif len(options.args) < 1: # print out the usages
     5                 sys.stdout.write(self.main_help_text()   'n')
     6             else: 
     7                 self.fetch_command(options.args[0]).print_help(self.prog_name, options.args[0])
     8         # Special-cases: We want 'django-admin --version' and
     9         # 'django-admin --help' to work, for backwards compatibility.
    10         elif subcommand == 'version' or self.argv[1:] == ['--version']:
    11             sys.stdout.write(django.get_version()   'n')
    12         elif self.argv[1:] in (['--help'], ['-h']):
    13             sys.stdout.write(self.main_help_text()   'n')
    14         else:
    15             self.fetch_command(subcommand).run_from_argv(self.argv)        
    

    最后看一眼fetch_command的代码:

     1     def fetch_command(self, subcommand):
     2         """
     3         Try to fetch the given subcommand, printing a message with the
     4         appropriate command called from the command line (usually
     5         "django-admin" or "manage.py") if it can't be found.
     6         """
     7         # Get commands outside of try block to prevent swallowing exceptions
     8         commands = get_commands()
     9         try:
    10             app_name = commands[subcommand]
    11         except KeyError:
    12             if os.environ.get('DJANGO_SETTINGS_MODULE'):
    13                 # If `subcommand` is missing due to misconfigured settings, the
    14                 # following line will retrigger an ImproperlyConfigured exception
    15                 # (get_commands() swallows the original one) so the user is
    16                 # informed about it.
    17                 settings.INSTALLED_APPS
    18             else:
    19                 sys.stderr.write("No Django settings specified.n")
    20             sys.stderr.write(
    21                 "Unknown command: %rnType '%s help' for usage.n"
    22                 % (subcommand, self.prog_name)
    23             )
    24             sys.exit(1)
    25         if isinstance(app_name, BaseCommand):
    26             # If the command is already loaded, use it directly.
    27             klass = app_name
    28         else:
    29             klass = load_command_class(app_name, subcommand)
    30         return klass
    

    这里主要用了load_command_class去导入相应的subcommand模块,并生成了一个command类对象。

    1 def load_command_class(app_name, name):
    2     """
    3     Given a command name and an application name, return the Command
    4     class instance. Allow all errors raised by the import process
    5     (ImportError, AttributeError) to propagate.
    6     """
    7     module = import_module('%s.management.commands.%s' % (app_name, name))
    8     #print("import %s %s" %(app_name, name))
    9     return module.Command()
    

    我们可以去django/core/management/command/目录下随便找一个command模块看一眼,比如说check.py,每个模块都有一个command类并继承自BaseCommand。上面提到的print_help方法或者run_from_argv方法都是在BaseCommand类中实现。

    1 class Command(BaseCommand):
    2     help = "Checks the entire Django project for potential problems."
    3 
    4     requires_system_checks = False
    5     
    6     #...
    

     

     


    以上是我的一点粗浅的理解,如果有觉得不对的地方,请多多指教,非常感谢!

    2018-03-21 17:57:17

     

    本文由新葡亰496net发布于奥门新萄京娱乐场,转载请注明出处:新葡亰496net命令系统,粗解之命令行执行

    关键词: