7x31
/
cmdb
mirror of https://github.com/veops/cmdb.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

前后端全面升级

This commit is contained in:
pycook 2023-07-10 17:42:15 +08:00
parent f57ff80099
commit 98cc853dbc
641 changed files with 97789 additions and 23995 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

700
LICENSE
View File

@ -1,125 +1,661 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 1989, 1991 Free Software Foundation, Inc.
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General Public License instead.) You can apply it to your programs, too.
Preamble
When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things.
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
The precise terms and conditions for copying, distribution and modification follow.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you".
The precise terms and conditions for copying, distribution and
modification follow.
Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does.
TERMS AND CONDITIONS
1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program.
0. Definitions.
You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee.
"This License" refers to version 3 of the GNU Affero General Public License.
2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions:
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License.
c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License.
A "covered work" means either the unmodified Program or a work based
on the Program.
3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following:
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.
1. Source Code.
5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License.
The Corresponding Source for a work in source code form is that
same work.
8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License.
2. Basic Permissions.
9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
NO WARRANTY
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.
4. Conveying Verbatim Copies.
To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
one line to give the program's name and an idea of what it does.
Copyright (C) yyyy name of author
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.
5. Conveying Modified Source Versions.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this when it starts in an interactive mode:
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details
type `show w'. This is free software, and you are welcome
to redistribute it under certain conditions; type `show c'
for details.
The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright
interest in the program `Gnomovision'
(which makes passes at compilers) written
by James Hacker.
signature of Ty Coon, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.

143
README.md
View File

@ -1,117 +1,70 @@
<h1 align="center">CMDB</h1>
<div align="center">
尽可能实现比较通用的运维资源的配置和管理
</div>
![基础资源视图](docs/logo.png)
<div align="center">
[![License](https://img.shields.io/badge/License-GPLv2-brightgreen)](https://github.com/pycook/cmdb/blob/master/LICENSE)
[![License](https://img.shields.io/badge/License-AGPLv3-brightgreen)](https://github.com/veops/cmdb/blob/master/LICENSE)
[![UI](https://img.shields.io/badge/UI-Ant%20Design%20Pro%20Vue-brightgreen)](https://github.com/sendya/ant-design-pro-vue)
[![API](https://img.shields.io/badge/API-Flask-brightgreen)](https://github.com/pallets/flask)
</div>
[English](README_en.md) / [中文](README.md)
- 在线预览: [CMDB](http://121.42.12.46:8000)
- 在线体验: <a href="https://cmdb.veops.cn" target="_blank">CMDB</a>
- username: demo
- password: 123456
> **重要提示**: `master` 分支在开发过程中可能处于 *不稳定的状态* 。
请通过[releases](https://github.com/pycook/cmdb/releases)获取
请通过[releases](https://github.com/veops/cmdb/releases)获取
Overview
----
系统介绍
-------------
### 整体架构
<img src=docs/view.png width=700 height=450 />
### 相关文档
- [设计文档](https://zhuanlan.zhihu.com/p/98453732)
- [API文档](https://github.com/pycook/cmdb/tree/master/docs)
- [树形视图实践](https://mp.weixin.qq.com/s/EflmmJ-qdUkddTx2hRt3pA)
- <a href="https://zhuanlan.zhihu.com/p/98453732" target="_blank">设计文档</a>
- <a href="https://github.com/veops/cmdb/tree/master/docs" target="_blank">API文档</a>
- <a href="https://mp.weixin.qq.com/s/EflmmJ-qdUkddTx2hRt3pA" target="_blank">树形视图实践</a>
### 3种类型视图
1. 资源视图 - 模型的实例数据, 用户可订阅
2. 树形视图 - 模型按字段分级, 用树形图方式展示, 用户可订阅
3. 关系视图 - 模型之间的关系, 用树形图方式展示, **管理员可配置**
##### 资源视图
![基础资源视图](https://raw.githubusercontent.com/pycook/cmdb/master/cmdb-ui/public/cmdb-ci.jpeg)
##### 树形视图
![树形视图](https://raw.githubusercontent.com/pycook/cmdb/master/cmdb-ui/public/cmdb-tree.jpeg)
##### 关系视图
![关系视图](https://raw.githubusercontent.com/pycook/cmdb/master/cmdb-ui/public/cmdb-relation.jpeg)
##### 用户订阅
![用户订阅](https://raw.githubusercontent.com/pycook/cmdb/master/cmdb-ui/public/cmdb-preference.jpeg)
##### 关系视图配置
![关系视图配置](https://raw.githubusercontent.com/pycook/cmdb/master/cmdb-ui/public/cmdb-relation-define.jpeg)
Docker一键快速构建
----
- 进入主目录（先安装docker环境）
```
docker-compose up -d
```
- 浏览器打开: [http://127.0.0.1:8000](http://127.0.0.1:8000)
本地搭建: 环境和依赖
----
- 存储: mysql, redis
- python版本: python2.7, >=python3.6
Install
----
- 启动mysql服务, redis服务
- 创建数据库cmdb
- 拉取代码
```bash
git clone https://github.com/pycook/cmdb.git
cd cmdb
cp cmdb-api/settings.py.example cmdb-api/settings.py
```
**设置cmdb-api/settings.py里的database**
- 安装库
- 后端: ```cd cmdb-api && pipenv run pipenv install && cd ..```
- 前端: ```cd cmdb-ui && yarn install && cd ..```
### 特点
- 灵活性
1. 规范并统一纳管复杂数据资产
2. 自动发现、入库IT资产
- 安全性
1. 细粒度访问控制
2. 完备操作日志
- 多应用
1. 丰富视图展示维度
2. 提供Restful API
3. 自定义字段触发器
- 创建数据库表: 进入**cmdb-api**目录执行 ```pipenv run flask db-setup && pipenv run flask init-cache```
- 可以将docs/cmdb.sql导入到数据库里，登录用户和密码分别是:demo/123456
- 启动服务
- 后端: 进入**cmdb-api**目录执行 ```pipenv run flask run -h 0.0.0.0```
- 前端: 进入**cmdb-ui**目录执行```yarn run serve```
- worker: 进入**cmdb-api**目录执行 ```pipenv run celery worker -A celery_worker.celery -E -Q cmdb_async --concurrency=1```
- 浏览器打开: [http://127.0.0.1:8000](http://127.0.0.1:8000)
- 如果是非本机访问, 要修改**cmdb-ui/.env**里**VUE_APP_API_BASE_URL**里的IP地址为后端服务的ip地址
### 主要功能
- 模型属性支持索引、多值、默认排序、字体颜色，支持计算属性
- 支持自动发现、定时巡检、文件导入
- 支持资源、树形、关系视图展示
- 支持模型间关系配置和展示
- 细粒度访问控制，完备的操作日志
- 支持跨模型搜索
### 系统概览
- 服务树
![1](docs/0.png "首页展示")
[查看更多展示](docs/screenshot.md)
### 更多功能
Install by Makefile
> 也欢迎移步[维易科技官网](https://www.veops.cn)，发现更多免费运维系统。
安装
----
- 启动mysql服务, redis服务
- 创建数据库cmdb
- 拉取代码
```bash
git clone https://github.com/pycook/cmdb.git
cd cmdb
cp cmdb-api/settings.py.example cmdb-api/settings.py
```
**设置cmdb-api/settings.py里的database**
- 顺序在cmdb目录下执行
- 环境: ```make env```
- 启动API: ```make api```
- 启动UI: ```make ui```
- 启动worker: ```make worker```
### [Docker一键快速构建](docs/docker.md)
### [本地搭建](docs/local.md)
### [Makefile安装](docs/makefile.md)
----
----------------------
_**欢迎加入CMDB运维开发QQ群（336164978）**_
![QQ群](cmdb-ui/public/qr_code.jpg)
![QQ群](docs/qr_code.jpg)

162
README_en.md
View File

@ -1,111 +1,66 @@
<h1 align="center">CMDB</h1>
<div align="center">
![基础资源视图](docs/logo.png)
As far as possible to achieve more universal configuration and management of IT resources
</div>
<div align="center">
[![License](https://img.shields.io/badge/License-GPLv2-brightgreen)](https://github.com/pycook/cmdb/blob/master/LICENSE)
[![License](https://img.shields.io/badge/License-AGPLv3-brightgreen)](https://github.com/veops/cmdb/blob/master/LICENSE)
[![UI](https://img.shields.io/badge/UI-Ant%20Design%20Pro%20Vue-brightgreen)](https://github.com/sendya/ant-design-pro-vue)
[![API](https://img.shields.io/badge/API-Flask-brightgreen)](https://github.com/pallets/flask)
</div>
[中文](README.md) / [English](README_en.md)
[English](README_en.md) / [中文](README.md)
## DEMO ONLINE
- Preview online: [CMDB](http://121.42.12.46:8000)
- Preview online: <a href="https://cmdb.veops.cn" target="_blank">CMDB</a>
- username: demo
- password: 123456
> **ATTENTION**: branch `master` may be unstable as the result of continued development, please pull code from [releases](https://github.com/pycook/cmdb/releases)
> **ATTENTION**: branch `master` may be unstable as the result of continued development, please pull code from [releases](https://github.com/veops/cmdb/releases)
Overview
-------------
### Technical Architecture
<img src=docs/view.png width=700 height=450 />
### Document
- <a href="https://zhuanlan.zhihu.com/p/98453732" target="_blank">Design Document</a>
- <a href="https://github.com/veops/cmdb/tree/master/docs" target="_blank">API Documentation</a>
- <a href="https://mp.weixin.qq.com/s/EflmmJ-qdUkddTx2hRt3pA" target="_blank">Practice of Tree View</a>
### Features
- Flexibility
1. Standardize and manage complex data assets
2. Automatically discover and inventory IT assets
- Security
1. Fine-grained access control
2. Comprehensive operation logs
- Multi-application
1. Rich view display dimensions
2. Provide Restful API
3. Custom field triggers
### Main Features
- Model attributes support indexing, multiple values, default sorting, font color, and computed properties.
- Support automatic discovery, scheduled inspections, and file import.
- Support resource, tree view, and relationship view display.
- Support configuration and display of relationships between models.
- Fine-grained access control and comprehensive operation logs.
- Support cross-model search.
### System Overview
- Service Tree
![1](docs/0.png "首页展示")
[View more screenshots](docs/screenshot.md)
### More Features
> Welcome to visit VeOps official website to discover more free operations and maintenance systems.
Installation
----
## Overview
### Documents
- [Design document](https://zhuanlan.zhihu.com/p/98453732)
- [API document](https://github.com/pycook/cmdb/tree/master/docs)
- [Tree view practice](https://mp.weixin.qq.com/s/EflmmJ-qdUkddTx2hRt3pA)
The CMDB is a universal project that can define and manage almost all IT resources, even every resource as long as you want to, which treat all IT resources as resource objects. Objects has both attributes and relationship.
CMDB's main distinguishing features as compared to other resource systems are:
- Define attributes of resource objects dynamically，you don't need to define all the attributes at the beginning.
- Define relationship of resource objects dynamically and simply, even you can draw the relationship through the web.
- Three view:
- Resource view: model instance data that users can subscribe
- Tree view: the model is hierarchical by field, shown in a tree diagram, and users can subscribe
- Relational view: relationships between models, shown in a tree diagram, **are configurable by the administrator**
- Authority management
### [One-Click Docker Quick Build](docs/docker_en.md)
### [Local Setup](docs/local_en.md)
### [Installation with Makefile](docs/makefile_en.md)
## Install
There are various ways of installing CMDB.
### Install by Docker
- Prepare: install docker and docker-compose
- In directory cmdb
```
docker-compose up -d
```
- View: [http://127.0.0.1:8000](http://127.0.0.1:8000)
### Environment and dependency
- database: mysql
- cache: redis
- python: python2.7, >=python3.6
### Install
- Start mysql, redis
- Create mysql database: cmdb
- Pull code
```bash
git clone https://github.com/pycook/cmdb.git
cd cmdb
cp cmdb-api/settings.py.example cmdb-api/settings.py
```
**set database in config file cmdb-api/settings.py**
- Install library
- backend: ```cd cmdb-api && pipenv run pipenv install && cd ..```
- frontend: ```cd cmdb-ui && yarn install && cd ..```
- Create tables of cmdb database:
in **cmdb-api** directory: ```pipenv run flask db-setup && pipenv run flask init-cache```
- Suggest step: (default: user:demo,password:123456)
``` source docs/cmdb.sql```
- Start service
- backend: in **cmdb-api** directory: ```pipenv run flask run -h 0.0.0.0```
- frontend: in **cmdb-ui** directory: ```yarn run serve```
- worker: in **cmdb-api** directory: ```pipenv run celery worker -A celery_worker.celery -E -Q cmdb_async --concurrency=1```
- homepage: [http://127.0.0.1:8000](http://127.0.0.1:8000)
- if not run localhost: please change ip address(**VUE_APP_API_BASE_URL**) in config file **cmdb-ui/.env** into your backend ip address
### Install by Makefile
- Start mysql,redis
- Create mysql database: cmdb
- Pull code
```bash
git clone https://github.com/pycook/cmdb.git
cd cmdb
cp cmdb-api/settings.py.example cmdb-api/settings.py
```
**set database in config file cmdb-api/settings.py**
- In cmdb directory,start in order as follows:
- enviroment: ```make env```
- start API: ```make api```
- start UI: ```make ui```
- start worker: ```make worker```
## Contributing
1. Fork it
@ -114,24 +69,7 @@ There are various ways of installing CMDB.
1. Push to the branch (`git push origin my-feature`)
1. Create new Pull Request
## DEMO
##### resource view
![resource view](https://raw.githubusercontent.com/pycook/cmdb/master/cmdb-ui/public/cmdb-ci.jpeg)
##### tree view
![tree view](https://raw.githubusercontent.com/pycook/cmdb/master/cmdb-ui/public/cmdb-tree.jpeg)
##### relationship view
![relationship view](https://raw.githubusercontent.com/pycook/cmdb/master/cmdb-ui/public/cmdb-relation.jpeg)
##### user subscription
![user subscription](https://raw.githubusercontent.com/pycook/cmdb/master/cmdb-ui/public/cmdb-preference.jpeg)
##### define relationship view
![define relationship view](https://raw.githubusercontent.com/pycook/cmdb/master/cmdb-ui/public/cmdb-relation-define.jpeg)
-----
_**Welcome to join us through QQ group（336164978）**_
![QQgroup](cmdb-ui/public/qr_code.jpg)
![QQgroup](docs/qr_code.jpg)

22
cmdb-api/Pipfile
View File

@ -6,7 +6,7 @@ name = "pypi"
[packages]
# Flask
Flask = "==1.0.3"
Werkzeug = ">=0.15.5"
Werkzeug = "==0.15.5"
click = ">=5.0"
# Api
Flask-RESTful = "==0.3.7"
@ -18,7 +18,6 @@ redis = "==3.2.1"
# Migrations
Flask-Migrate = "==2.5.2"
# Deployment
gevent = "==1.4.0"
gunicorn = "==19.5.0"
supervisor = "==4.0.3"
# Auth
@ -26,6 +25,7 @@ Flask-Login = "==0.4.1"
Flask-Bcrypt = "==0.7.1"
Flask-Cors = ">=3.0.8"
python-ldap = "==3.2.0"
pycryptodome = "==3.12.0"
# Caching
Flask-Caching = ">=1.0.0"
# Environment variable parsing
@ -33,16 +33,32 @@ environs = "==4.2.0"
marshmallow = "==2.20.2"
# async tasks
celery = "==4.3.0"
celery_once = "==3.0.1"
more-itertools = "==5.0.0"
kombu = "==4.4.0"
# common setting
Flask-APScheduler = "==1.12.4"
timeout-decorator = "==0.5.0"
numpy = "==1.18.5"
pandas = "==1.3.2"
WTForms = "==3.0.0"
email-validator = "==1.3.1"
treelib = "==1.6.1"
flasgger = "==0.9.5"
Pillow = "==8.3.2"
# other
six = "==1.12.0"
bs4 = ">=0.0.1"
toposort = ">=1.5"
requests = ">=2.22.0"
PyJWT = ">=1.7.1"
elasticsearch = "==7.0.4"
elasticsearch = "==7.17.9"
future = "==0.18.2"
itsdangerous = "==2.0.1"
Jinja2 = "==3.0.1"
jinja2schema = "==0.1.4"
msgpack-python = "==0.5.6"
alembic = "==1.7.7"
[dev-packages]
# Testing

0
cmdb-ui/src/views/welcome/welcome.vue → cmdb-api/README.md
View File

62
cmdb-api/api/app.py
View File

@ -1,18 +1,20 @@
# -*- coding: utf-8 -*-
"""The app module, containing the app factory function."""
import datetime
import decimal
import logging
import os
import sys
from inspect import getmembers
from logging.handlers import RotatingFileHandler
from api.flask_cas import CAS
from flask import Flask
from flask import make_response, jsonify
from flask.blueprints import Blueprint
from flask.cli import click
from flask.json import JSONEncoder
import api.views
import api.views.entry
from api.extensions import (
bcrypt,
cors,
@ -22,9 +24,10 @@ from api.extensions import (
migrate,
celery,
rd,
es
es,
)
from .models.acl import User
from api.flask_cas import CAS
from api.models.acl import User
HERE = os.path.abspath(os.path.dirname(__file__))
PROJECT_ROOT = os.path.join(HERE, os.pardir)
@ -34,7 +37,7 @@ API_PACKAGE = "api"
@login_manager.user_loader
def load_user(user_id):
"""Load user by ID."""
return User.get_by(uid=int(user_id), first=True, to_dict=False)
return User.get_by_id(int(user_id))
class ReverseProxy(object):
@ -72,24 +75,59 @@ class ReverseProxy(object):
return self.app(environ, start_response)
class MyJSONEncoder(JSONEncoder):
def default(self, o):
if isinstance(o, (decimal.Decimal, datetime.date, datetime.time)):
return str(o)
if isinstance(o, datetime.datetime):
return o.strftime('%Y-%m-%d %H:%M:%S')
return o
def create_acl_app(config_object="settings"):
app = Flask(__name__.split(".")[0])
app.config.from_object(config_object)
register_extensions(app)
return app
def create_app(config_object="settings"):
"""Create application factory, as explained here: http://flask.pocoo.org/docs/patterns/appfactories/.
:param config_object: The configuration object to use.
"""
app = Flask(__name__.split(".")[0])
app.config.from_object(config_object)
app.json_encoder = MyJSONEncoder
configure_logger(app)
register_extensions(app)
register_blueprints(app)
register_error_handlers(app)
register_shell_context(app)
register_commands(app)
configure_logger(app)
CAS(app)
app.wsgi_app = ReverseProxy(app.wsgi_app)
configure_upload_dir(app)
return app
def configure_upload_dir(app):
upload_dir = app.config.get('UPLOAD_DIRECTORY', 'uploaded_files')
common_setting_path = os.path.join(HERE, upload_dir)
for path in [common_setting_path]:
if not os.path.exists(path):
app.logger.warning(f"{path}, not exist, create...")
os.makedirs(path)
app.config['UPLOAD_DIRECTORY_FULL'] = common_setting_path
def register_extensions(app):
"""Register Flask extensions."""
bcrypt.init_app(app)
@ -99,13 +137,13 @@ def register_extensions(app):
login_manager.init_app(app)
migrate.init_app(app, db)
rd.init_app(app)
if app.config.get("USE_ES"):
if app.config.get('USE_ES'):
es.init_app(app)
celery.conf.update(app.config)
def register_blueprints(app):
for item in getmembers(api.views):
for item in getmembers(api.views.entry):
if item[0].startswith("blueprint") and isinstance(item[1], Blueprint):
app.register_blueprint(item[1])
@ -118,10 +156,16 @@ def register_error_handlers(app):
import traceback
app.logger.error(traceback.format_exc())
error_code = getattr(error, "code", 500)
return make_response(jsonify(message=str(error)), error_code)
if not str(error_code).isdigit():
error_code = 400
if error_code != 500:
return make_response(jsonify(message=str(error)), error_code)
else:
return make_response(jsonify(message=traceback.format_exc(-1)), error_code)
for errcode in app.config.get("ERROR_CODES") or [400, 401, 403, 404, 405, 500, 502]:
app.errorhandler(errcode)(render_error)
app.handle_exception = render_error

69
cmdb-api/api/commands/click_acl.py Normal file
View File

@ -0,0 +1,69 @@
import click
from flask.cli import with_appcontext
@click.command()
@with_appcontext
def init_acl():
from api.models.acl import Role
from api.models.acl import App
from api.tasks.acl import role_rebuild
from api.lib.perm.acl.const import ACL_QUEUE
roles = Role.get_by(to_dict=False)
apps = App.get_by(to_dict=False)
for role in roles:
if role.app_id:
role_rebuild.apply_async(args=(role.id, role.app_id), queue=ACL_QUEUE)
else:
for app in apps:
role_rebuild.apply_async(args=(role.id, app.id), queue=ACL_QUEUE)
# @click.command()
# @with_appcontext
# def acl_clean():
# from api.models.acl import Resource
# from api.models.acl import Permission
# from api.models.acl import RolePermission
#
# perms = RolePermission.get_by(to_dict=False)
#
# for r in perms:
# perm = Permission.get_by_id(r.perm_id)
# if perm and perm.app_id != r.app_id:
# resource_id = r.resource_id
# resource = Resource.get_by_id(resource_id)
# perm_name = perm.name
# existed = Permission.get_by(resource_type_id=resource.resource_type_id, name=perm_name, first=True,
# to_dict=False)
# if existed is not None:
# other = RolePermission.get_by(rid=r.rid, perm_id=existed.id, resource_id=resource_id)
# if not other:
# r.update(perm_id=existed.id)
# else:
# r.soft_delete()
# else:
# r.soft_delete()
#
#
# @click.command()
# @with_appcontext
# def acl_has_resource_role():
# from api.models.acl import Role
# from api.models.acl import App
# from api.lib.perm.acl.cache import HasResourceRoleCache
# from api.lib.perm.acl.role import RoleCRUD
#
# roles = Role.get_by(to_dict=False)
# apps = App.get_by(to_dict=False)
# for role in roles:
# if role.app_id:
# res = RoleCRUD.recursive_resources(role.id, role.app_id)
# if res.get('resources') or res.get('groups'):
# HasResourceRoleCache.add(role.id, role.app_id)
# else:
# for app in apps:
# res = RoleCRUD.recursive_resources(role.id, app.id)
# if res.get('resources') or res.get('groups'):
# HasResourceRoleCache.add(role.id, app.id)

74
cmdb-api/api/commands/click_cmdb.py
View File

@ -1,7 +1,10 @@
# -*- coding:utf-8 -*-
import copy
import datetime
import json
import time
import click
from flask import current_app
@ -10,6 +13,7 @@ from flask.cli import with_appcontext
import api.lib.cmdb.ci
from api.extensions import db
from api.extensions import rd
from api.lib.cmdb.ci_type import CITypeTriggerManager
from api.lib.cmdb.const import PermEnum
from api.lib.cmdb.const import REDIS_PREFIX_CI
from api.lib.cmdb.const import REDIS_PREFIX_CI_RELATION
@ -28,12 +32,13 @@ from api.models.acl import ResourceType
from api.models.cmdb import CI
from api.models.cmdb import CIRelation
from api.models.cmdb import CIType
from api.models.cmdb import CITypeTrigger
from api.models.cmdb import PreferenceRelationView
@click.command()
@with_appcontext
def init_cache():
def cmdb_init_cache():
db.session.remove()
if current_app.config.get("USE_ES"):
@ -92,7 +97,7 @@ def init_cache():
@click.command()
@with_appcontext
def init_acl():
def cmdb_init_acl():
_app = AppCache.get('cmdb') or App.create(name='cmdb')
app_id = _app.id
@ -173,7 +178,6 @@ def add_user(user, password, mail, is_admin):
assert user is not None
assert password is not None
assert mail is not None
print((user, password, is_admin))
UserCRUD.add(username=user, password=password, email=mail, is_admin=is_admin)
@ -195,3 +199,67 @@ def del_user(user):
u = User.get_by(username=user, first=True, to_dict=False)
u and UserCRUD.delete(u.uid)
@click.command()
@with_appcontext
def cmdb_counter():
from api.lib.cmdb.cache import CMDBCounterCache
while True:
try:
db.session.remove()
CMDBCounterCache.reset()
except:
import traceback
print(traceback.format_exc())
time.sleep(60)
@click.command()
@with_appcontext
def cmdb_trigger():
current_day = datetime.datetime.today().strftime("%Y-%m-%d")
trigger2cis = dict()
trigger2completed = dict()
i = 0
while True:
db.session.remove()
if datetime.datetime.today().strftime("%Y-%m-%d") != current_day:
trigger2cis = dict()
trigger2completed = dict()
current_day = datetime.datetime.today().strftime("%Y-%m-%d")
if i == 360 or i == 0:
i = 0
try:
triggers = CITypeTrigger.get_by(to_dict=False)
for trigger in triggers:
ready_cis = CITypeTriggerManager.waiting_cis(trigger)
if trigger.id not in trigger2cis:
trigger2cis[trigger.id] = (trigger, ready_cis)
else:
cur = trigger2cis[trigger.id]
cur_ci_ids = {i.ci_id for i in cur[1]}
trigger2cis[trigger.id] = (trigger, cur[1] + [i for i in ready_cis if i.ci_id not in cur_ci_ids
and i.ci_id not in trigger2completed[trigger.id]])
except Exception as e:
print(e)
for tid in trigger2cis:
trigger, cis = trigger2cis[tid]
for ci in copy.deepcopy(cis):
if CITypeTriggerManager.trigger_notify(trigger, ci):
trigger2completed.setdefault(trigger.id, set()).add(ci.ci_id)
for _ci in cis:
if _ci.ci_id == ci.ci_id:
cis.remove(_ci)
i += 1
time.sleep(10)

4
cmdb-api/api/commands/common.py
View File

@ -78,7 +78,7 @@ def clean():
"""
for dirpath, dirnames, filenames in os.walk("."):
for filename in filenames:
if filename.endswith(".pyc") or filename.endswith(".pyo"):
if filename.endswith(".pyc") or filename.endswith(".pyo") or filename.endswith(".c"):
full_pathname = os.path.join(dirpath, filename)
click.echo("Removing {}".format(full_pathname))
os.remove(full_pathname)
@ -110,7 +110,7 @@ def urls(url, order):
column_length = 1
else:
rules = sorted(
current_app.url_map.iter_rules(), key=lambda x: getattr(x, order)
current_app.url_map.iter_rules(), key=lambda rule: getattr(rule, order)
)
for rule in rules:
rows.append((rule.rule, rule.endpoint, None))

164
cmdb-api/api/commands/init_common_setting.py Normal file
View File

@ -0,0 +1,164 @@
import click
from flask import current_app
from flask.cli import with_appcontext
from werkzeug.datastructures import MultiDict
from api.lib.common_setting.acl import ACLManager
from api.lib.common_setting.employee import EmployeeAddForm
from api.lib.common_setting.resp_format import ErrFormat
from api.models.common_setting import Employee, Department
class InitEmployee(object):
"""
初始化员工
"""
def __init__(self):
self.log = current_app.logger
def import_user_from_acl(self):
"""
从ACL导入用户
"""
acl = ACLManager('acl')
user_list = acl.get_all_users()
username_list = [e['username'] for e in Employee.get_by()]
for user in user_list:
if user['username'] in username_list:
continue
try:
form = EmployeeAddForm(MultiDict(user))
if not form.validate():
raise Exception(
','.join(['{}: {}'.format(filed, ','.join(msg)) for filed, msg in form.errors.items()]))
data = form.data
data['acl_uid'] = user['uid']
data['block'] = 1 if user['block'] else 0
data.pop('password')
Employee.create(
**data
)
except Exception as e:
self.log.error(ErrFormat.acl_import_user_failed.format(user['username'], str(e)))
self.log.error(e)
class InitDepartment(object):
def __init__(self):
self.log = current_app.logger
def init(self):
self.init_wide_company()
def hard_delete(self, department_id, department_name):
existed_deleted_list = Department.query.filter(
Department.department_name == department_name,
Department.department_id == department_id,
Department.deleted == 1,
).all()
for existed in existed_deleted_list:
existed.delete()
def get_department(self, department_name):
return Department.query.filter(
Department.department_name == department_name,
Department.deleted == 0,
).order_by(Department.created_at.asc()).first()
def run(self, department_id, department_name, department_parent_id):
self.hard_delete(department_id, department_name)
res = self.get_department(department_name)
if res:
if res.department_id == department_id:
return
else:
new_d = res.update(
department_id=department_id,
department_parent_id=department_parent_id,
)
return
Department.create(
department_id=department_id,
department_name=department_name,
department_parent_id=department_parent_id,
)
new_d = self.get_department(department_name)
if new_d.department_id != department_id:
new_d = new_d.update(
department_id=department_id,
department_parent_id=department_parent_id,
)
self.log.info(f"初始化 {department_name} 部门成功.")
def run_common(self, department_id, department_name, department_parent_id):
try:
self.run(department_id, department_name, department_parent_id)
except Exception as e:
current_app.logger.error(f"init {department_name} err:")
current_app.logger.error(e)
raise Exception(e)
def init_wide_company(self):
"""
创建 id 0, name 全公司 的部门
"""
department_id = 0
department_name = '全公司'
department_parent_id = -1
self.run_common(department_id, department_name, department_parent_id)
def create_acl_role_with_department(self):
"""
当前所有部门，在ACL创建 role
"""
acl = ACLManager('acl')
role_name_map = {role['name']: role for role in acl.get_all_roles()}
d_list = Department.query.filter(
Department.deleted == 0, Department.department_parent_id != -1).all()
for department in d_list:
if department.acl_rid > 0:
continue
role = role_name_map.get(department.department_name)
if role is None:
payload = {
'app_id': 'acl',
'name': department.department_name,
}
role = acl.create_role(payload)
acl_rid = role.get('id') if role else 0
department.update(
acl_rid=acl_rid
)
info = f"update department acl_rid: {acl_rid}"
current_app.logger.info(info)
@click.command()
@with_appcontext
def init_import_user_from_acl():
"""
从ACL导入用户
"""
InitEmployee().import_user_from_acl()
@click.command()
@with_appcontext
def init_department():
"""
初始化 部门
"""
InitDepartment().init()
InitDepartment().create_acl_role_with_department()

2
cmdb-api/api/extensions.py
View File

@ -14,7 +14,7 @@ from api.lib.utils import RedisHandler
bcrypt = Bcrypt()
login_manager = LoginManager()
db = SQLAlchemy()
db = SQLAlchemy(session_options={"autoflush": False})
migrate = Migrate()
cache = Cache()
celery = Celery()

2
cmdb-api/api/flask_cas/__init__.py
View File

@ -75,4 +75,4 @@ class CAS(object):
@property
def token(self):
return flask.session.get(
self.app.config['CAS_TOKEN_SESSION_KEY'], None)
self.app.config['CAS_TOKEN_SESSION_KEY'], None)

6
cmdb-api/api/flask_cas/cas_urls.py
View File

@ -68,7 +68,7 @@ def create_cas_login_url(cas_url, cas_route, service,
('service', service),
('renew', renew),
('gateway', gateway),
)
)
def create_cas_logout_url(cas_url, cas_route, url=None):
@ -91,7 +91,7 @@ def create_cas_logout_url(cas_url, cas_route, url=None):
cas_url,
cas_route,
('service', url),
)
)
def create_cas_validate_url(cas_url, cas_route, service, ticket,
@ -119,4 +119,4 @@ def create_cas_validate_url(cas_url, cas_route, service, ticket,
('service', service),
('ticket', ticket),
('renew', renew),
)
)

13
cmdb-api/api/flask_cas/routing.py
View File

@ -132,15 +132,20 @@ def validate(ticket):
current_app.logger.debug("valid")
session[cas_username_session_key] = username
user = UserCache.get(username)
session["acl"] = dict(uid=user_info.get("uuid"),
from api.lib.perm.acl.acl import ACLManager
user_info = ACLManager.get_user_info(username)
session["acl"] = dict(uid=user_info.get("uid"),
avatar=user.avatar if user else user_info.get("avatar"),
userId=user_info.get("id"),
userName=user_info.get("name"),
userId=user_info.get("uid"),
rid=user_info.get("rid"),
userName=user_info.get("username"),
nickName=user_info.get("nickname"),
parentRoles=user_info.get("parents"),
childRoles=user_info.get("children"),
roleName=user_info.get("role"))
session["uid"] = user_info.get("uuid")
session["uid"] = user_info.get("uid")
current_app.logger.debug(session)
current_app.logger.debug(request.url)
else:

197
cmdb-api/api/lib/cmdb/attribute.py
View File

@ -1,13 +1,22 @@
# -*- coding:utf-8 -*-
import requests
from flask import abort
from flask import current_app
from flask import g
from flask import session
from api.extensions import db
from api.lib.cmdb.cache import AttributeCache
from api.lib.cmdb.const import CITypeOperateType
from api.lib.cmdb.const import ResourceTypeEnum, RoleEnum, PermEnum
from api.lib.cmdb.const import ValueTypeEnum
from api.lib.cmdb.history import CITypeHistoryManager
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.cmdb.utils import ValueTypeMap
from api.lib.decorator import kwargs_required
from api.lib.perm.acl.acl import is_app_admin
from api.lib.perm.acl.acl import validate_permission
from api.models.cmdb import Attribute
from api.models.cmdb import CITypeAttribute
from api.models.cmdb import CITypeAttributeGroupItem
@ -18,27 +27,63 @@ class AttributeManager(object):
"""
CI attributes manager
"""
cls = Attribute
def __init__(self):
pass
@staticmethod
def get_choice_values(attr_id, value_type):
def _get_choice_values_from_web_hook(choice_web_hook):
url = choice_web_hook.get('url')
ret_key = choice_web_hook.get('ret_key')
headers = choice_web_hook.get('headers') or {}
payload = choice_web_hook.get('payload') or {}
method = choice_web_hook.get('method', 'GET').lower()
try:
res = getattr(requests, method)(url, headers=headers, data=payload).json()
if ret_key:
ret_key_list = ret_key.strip().split("##")
for key in ret_key_list[:-1]:
if key in res:
res = res[key]
if isinstance(res, list):
return [[i[ret_key_list[-1]], {}] for i in res if i.get(ret_key_list[-1])]
return [[i, {}] for i in (res.get(ret_key_list[-1]) or [])]
except Exception as e:
current_app.logger.error(str(e))
return []
@classmethod
def get_choice_values(cls, attr_id, value_type, choice_web_hook, choice_web_hook_parse=True):
if choice_web_hook and isinstance(choice_web_hook, dict) and choice_web_hook_parse:
return cls._get_choice_values_from_web_hook(choice_web_hook)
elif choice_web_hook and not choice_web_hook_parse:
return []
choice_table = ValueTypeMap.choice.get(value_type)
choice_values = choice_table.get_by(fl=["value"], attr_id=attr_id)
return [choice_value["value"] for choice_value in choice_values]
choice_values = choice_table.get_by(fl=["value", "option"], attr_id=attr_id)
return [[choice_value['value'], choice_value['option']] for choice_value in choice_values]
@staticmethod
def _add_choice_values(_id, value_type, choice_values):
def add_choice_values(_id, value_type, choice_values):
choice_table = ValueTypeMap.choice.get(value_type)
db.session.query(choice_table).filter(choice_table.attr_id == _id).delete()
db.session.flush()
choice_values = choice_values
for v in choice_values:
table = choice_table(attr_id=_id, value=v)
for v, option in choice_values:
table = choice_table(attr_id=_id, value=v, option=option)
db.session.add(table)
db.session.flush()
try:
db.session.flush()
except:
return abort(400, ErrFormat.invalid_choice_values)
@staticmethod
def _del_choice_values(_id, value_type):
@ -67,7 +112,10 @@ class AttributeManager(object):
attrs = attrs[(page - 1) * page_size:][:page_size]
res = list()
for attr in attrs:
attr["is_choice"] and attr.update(dict(choice_value=cls.get_choice_values(attr["id"], attr["value_type"])))
attr["is_choice"] and attr.update(dict(choice_value=cls.get_choice_values(
attr["id"], attr["value_type"], attr["choice_web_hook"])))
attr['is_choice'] and attr.pop('choice_web_hook', None)
res.append(attr)
return numfound, res
@ -75,54 +123,72 @@ class AttributeManager(object):
def get_attribute_by_name(self, name):
attr = Attribute.get_by(name=name, first=True)
if attr and attr["is_choice"]:
attr.update(dict(choice_value=self.get_choice_values(attr["id"], attr["value_type"])))
attr.update(dict(choice_value=self.get_choice_values(
attr["id"], attr["value_type"], attr["choice_web_hook"])))
return attr
def get_attribute_by_alias(self, alias):
attr = Attribute.get_by(alias=alias, first=True)
if attr and attr["is_choice"]:
attr.update(dict(choice_value=self.get_choice_values(attr["id"], attr["value_type"])))
attr.update(dict(choice_value=self.get_choice_values(
attr["id"], attr["value_type"], attr["choice_web_hook"])))
return attr
def get_attribute_by_id(self, _id):
attr = Attribute.get_by_id(_id).to_dict()
if attr and attr["is_choice"]:
attr.update(dict(choice_value=self.get_choice_values(attr["id"], attr["value_type"])))
attr.update(dict(choice_value=self.get_choice_values(
attr["id"], attr["value_type"], attr["choice_web_hook"])))
return attr
def get_attribute(self, key):
def get_attribute(self, key, choice_web_hook_parse=True):
attr = AttributeCache.get(key).to_dict()
if attr and attr["is_choice"]:
attr.update(dict(choice_value=self.get_choice_values(attr["id"], attr["value_type"])))
attr.update(dict(choice_value=self.get_choice_values(
attr["id"], attr["value_type"], attr["choice_web_hook"])), choice_web_hook_parse=choice_web_hook_parse)
return attr
@staticmethod
def can_create_computed_attribute():
if RoleEnum.CONFIG not in session.get("acl", {}).get("parentRoles", []) and not is_app_admin('cmdb'):
return abort(403, ErrFormat.role_required.format(RoleEnum.CONFIG))
@classmethod
@kwargs_required("name")
def add(cls, **kwargs):
choice_value = kwargs.pop("choice_value", [])
kwargs.pop("is_choice", None)
is_choice = True if choice_value else False
is_choice = True if choice_value or kwargs.get('choice_web_hook') else False
name = kwargs.pop("name")
if name in {'id', '_id', 'ci_id', 'type', '_type', 'ci_type'}:
return abort(400, ErrFormat.attribute_name_cannot_be_builtin)
alias = kwargs.pop("alias", "")
alias = name if not alias else alias
Attribute.get_by(name=name, first=True) and abort(400, "attribute name <{0}> is duplicated".format(name))
Attribute.get_by(alias=alias, first=True) and abort(400, "attribute alias <{0}> is duplicated".format(name))
Attribute.get_by(name=name, first=True) and abort(400, ErrFormat.attribute_name_duplicate.format(name))
if kwargs.get('default') and not (isinstance(kwargs['default'], dict) and 'default' in kwargs['default']):
kwargs['default'] = dict(default=kwargs['default'])
kwargs.get('is_computed') and cls.can_create_computed_attribute()
attr = Attribute.create(flush=True,
name=name,
alias=alias,
is_choice=is_choice,
uid=g.user.uid,
**kwargs)
if choice_value:
cls._add_choice_values(attr.id, attr.value_type, choice_value)
cls.add_choice_values(attr.id, attr.value_type, choice_value)
try:
db.session.commit()
except Exception as e:
db.session.rollback()
current_app.logger.error("add attribute error, {0}".format(str(e)))
return abort(400, "add attribute <{0}> failed".format(name))
return abort(400, ErrFormat.add_attribute_failed.format(name))
AttributeCache.clean(attr)
@ -144,27 +210,86 @@ class AttributeManager(object):
return attr.id
@staticmethod
def _change_index(attr, old, new):
from api.lib.cmdb.utils import TableMap
from api.tasks.cmdb import batch_ci_cache
from api.lib.cmdb.const import CMDB_QUEUE
old_table = TableMap(attr=attr, is_index=old).table
new_table = TableMap(attr=attr, is_index=new).table
ci_ids = []
for i in db.session.query(old_table).filter(getattr(old_table, 'attr_id') == attr.id):
new_table.create(ci_id=i.ci_id, attr_id=attr.id, value=i.value, flush=True)
ci_ids.append(i.ci_id)
db.session.query(old_table).filter(getattr(old_table, 'attr_id') == attr.id).delete()
try:
db.session.commit()
except Exception as e:
db.session.rollback()
current_app.logger.error(str(e))
return abort(400, ErrFormat.attribute_index_change_failed)
batch_ci_cache.apply_async(args=(ci_ids,), queue=CMDB_QUEUE)
@staticmethod
def _can_edit_attribute(attr):
from api.lib.cmdb.ci_type import CITypeManager
if attr.uid == g.user.uid:
return True
for i in CITypeAttribute.get_by(attr_id=attr.id, to_dict=False):
resource = CITypeManager.get_name_by_id(i.type_id)
if resource:
validate_permission(resource, ResourceTypeEnum.CI, PermEnum.CONFIG, "cmdb")
return True
def update(self, _id, **kwargs):
attr = Attribute.get_by_id(_id) or abort(404, "Attribute <{0}> does not exist".format(_id))
attr = Attribute.get_by_id(_id) or abort(404, ErrFormat.attribute_not_found.format("id={}".format(_id)))
if not self._can_edit_attribute(attr):
return abort(403, ErrFormat.cannot_edit_attribute)
if kwargs.get("name"):
other = Attribute.get_by(name=kwargs['name'], first=True, to_dict=False)
if other and other.id != attr.id:
return abort(400, "Attribute name <{0}> cannot be duplicate!".format(kwargs['name']))
if kwargs.get("alias"):
other = Attribute.get_by(alias=kwargs['alias'], first=True, to_dict=False)
if other and other.id != attr.id:
return abort(400, "Attribute alias <{0}> cannot be duplicate!".format(kwargs['alias']))
return abort(400, ErrFormat.attribute_name_duplicate.format(kwargs['name']))
if attr.value_type != kwargs.get('value_type'):
return abort(400, ErrFormat.attribute_value_type_cannot_change)
if "is_list" in kwargs and kwargs['is_list'] != attr.is_list:
return abort(400, ErrFormat.attribute_list_value_cannot_change)
if "is_index" in kwargs and kwargs['is_index'] != attr.is_index:
if not is_app_admin("cmdb"):
return abort(400, ErrFormat.attribute_index_cannot_change)
self._change_index(attr, attr.is_index, kwargs['is_index'])
existed2 = attr.to_dict()
if not existed2['choice_web_hook'] and existed2['is_choice']:
existed2['choice_value'] = self.get_choice_values(attr.id, attr.value_type, attr.choice_web_hook)
choice_value = kwargs.pop("choice_value", False)
is_choice = True if choice_value else False
is_choice = True if choice_value or kwargs.get('choice_web_hook') else False
kwargs['is_choice'] = is_choice
attr.update(flush=True, **kwargs)
if kwargs.get('default') and not (isinstance(kwargs['default'], dict) and 'default' in kwargs['default']):
kwargs['default'] = dict(default=kwargs['default'])
if is_choice:
self._add_choice_values(attr.id, attr.value_type, choice_value)
else:
kwargs.get('is_computed') and self.can_create_computed_attribute()
attr.update(flush=True, filter_none=False, **kwargs)
if is_choice and choice_value:
self.add_choice_values(attr.id, attr.value_type, choice_value)
elif is_choice:
self._del_choice_values(attr.id, attr.value_type)
try:
@ -172,7 +297,14 @@ class AttributeManager(object):
except Exception as e:
db.session.rollback()
current_app.logger.error("update attribute error, {0}".format(str(e)))
return abort(400, "update attribute <{0}> failed".format(_id))
return abort(400, ErrFormat.update_attribute_failed.format(("id=".format(_id))))
new = attr.to_dict()
if not new['choice_web_hook'] and new['is_choice']:
new['choice_value'] = choice_value
CITypeHistoryManager.add(CITypeOperateType.UPDATE_ATTRIBUTE, None, attr_id=attr.id,
change=dict(old=existed2, new=new))
AttributeCache.clean(attr)
@ -180,9 +312,12 @@ class AttributeManager(object):
@staticmethod
def delete(_id):
attr = Attribute.get_by_id(_id) or abort(404, "Attribute <{0}> does not exist".format(_id))
attr = Attribute.get_by_id(_id) or abort(404, ErrFormat.attribute_not_found.format("id={}".format(_id)))
name = attr.name
if attr.uid and attr.uid != g.user.uid:
return abort(403, ErrFormat.cannot_delete_attribute)
if attr.is_choice:
choice_table = ValueTypeMap.choice.get(attr.value_type)
db.session.query(choice_table).filter(choice_table.attr_id == _id).delete() # FIXME: session conflict

1
cmdb-api/api/lib/cmdb/auto_discovery/__init__.py Normal file
View File

@ -0,0 +1 @@
# -*- coding:utf-8 -*-

505
cmdb-api/api/lib/cmdb/auto_discovery/auto_discovery.py Normal file
View File

@ -0,0 +1,505 @@
# -*- coding:utf-8 -*-
import datetime
import json
import os
from flask import abort
from flask import current_app
from flask import g
from sqlalchemy import func
from api.extensions import db
from api.lib.cmdb.auto_discovery.const import ClOUD_MAP
from api.lib.cmdb.cache import CITypeAttributeCache
from api.lib.cmdb.cache import CITypeCache
from api.lib.cmdb.ci import CIManager
from api.lib.cmdb.ci import CIRelationManager
from api.lib.cmdb.ci_type import CITypeGroupManager
from api.lib.cmdb.const import AutoDiscoveryType
from api.lib.cmdb.const import PermEnum
from api.lib.cmdb.const import ResourceTypeEnum
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.cmdb.search import SearchError
from api.lib.cmdb.search.ci import search
from api.lib.mixin import DBMixin
from api.lib.perm.acl.acl import is_app_admin
from api.lib.perm.acl.acl import validate_permission
from api.lib.utils import AESCrypto
from api.models.cmdb import AutoDiscoveryCI
from api.models.cmdb import AutoDiscoveryCIType
from api.models.cmdb import AutoDiscoveryRule
PWD = os.path.abspath(os.path.dirname(__file__))
def parse_plugin_script(script):
attributes = []
try:
x = compile(script, '', "exec")
exec(x)
unique_key = locals()['AutoDiscovery']().unique_key
attrs = locals()['AutoDiscovery']().attributes() or []
except Exception as e:
return abort(400, str(e))
if not isinstance(attrs, list):
return abort(400, ErrFormat.adr_plugin_attributes_list_required)
for i in attrs:
if len(i) == 3:
name, _type, desc = i
elif len(i) == 2:
name, _type = i
desc = ""
else:
continue
attributes.append(dict(name=name, type=_type, desc=desc))
return unique_key, attributes
def check_plugin_script(**kwargs):
kwargs['unique_key'], kwargs['attributes'] = parse_plugin_script(kwargs['plugin_script'])
if not kwargs.get('unique_key'):
return abort(400, ErrFormat.adr_unique_key_required)
if not kwargs.get('attributes'):
return abort(400, ErrFormat.adr_plugin_attributes_list_no_empty)
return kwargs
class AutoDiscoveryRuleCRUD(DBMixin):
cls = AutoDiscoveryRule
@classmethod
def get_by_name(cls, name):
return cls.cls.get_by(name=name, first=True, to_dict=False)
@classmethod
def get_by_id(cls, _id):
return cls.cls.get_by_id(_id)
def get_by_inner(self):
return self.cls.get_by(is_inner=True, to_dict=True)
def import_template(self, rules):
for rule in rules:
rule.pop("id", None)
rule.pop("created_at", None)
rule.pop("updated_at", None)
existed = self.cls.get_by(name=rule['name'], first=True, to_dict=False)
if existed is not None:
existed.update(**rule)
else:
self.cls.create(**rule)
def _can_add(self, **kwargs):
self.cls.get_by(name=kwargs['name']) and abort(400, ErrFormat.adr_duplicate.format(kwargs['name']))
if kwargs.get('is_plugin') and kwargs.get('plugin_script'):
kwargs = check_plugin_script(**kwargs)
return kwargs
def _can_update(self, **kwargs):
existed = self.cls.get_by_id(kwargs['_id']) or abort(
404, ErrFormat.adr_not_found.format("id={}".format(kwargs['_id'])))
if 'name' in kwargs and not kwargs['name']:
return abort(400, ErrFormat.argument_value_required.format('name'))
if kwargs.get('name'):
other = self.cls.get_by(name=kwargs['name'], first=True, to_dict=False)
if other and other.id != existed.id:
return abort(400, ErrFormat.adr_duplicate.format(kwargs['name']))
return existed
def update(self, _id, **kwargs):
if kwargs.get('is_plugin') and kwargs.get('plugin_script'):
kwargs = check_plugin_script(**kwargs)
return super(AutoDiscoveryRuleCRUD, self).update(_id, filter_none=False, **kwargs)
def _can_delete(self, **kwargs):
if AutoDiscoveryCIType.get_by(adr_id=kwargs['_id'], first=True):
return abort(400, ErrFormat.adr_referenced)
return self._can_update(**kwargs)
class AutoDiscoveryCITypeCRUD(DBMixin):
cls = AutoDiscoveryCIType
@classmethod
def get_all(cls):
return cls.cls.get_by(to_dict=False)
@classmethod
def get_by_id(cls, _id):
return cls.cls.get_by_id(_id)
@classmethod
def get_by_type_id(cls, type_id):
return cls.cls.get_by(type_id=type_id, to_dict=False)
@classmethod
def get(cls, ci_id, oneagent_id, last_update_at=None):
result = []
rules = cls.cls.get_by(to_dict=True)
for rule in rules:
if rule.get('relation'):
continue
if isinstance(rule.get("extra_option"), dict) and rule['extra_option'].get('secret'):
if not (g.user.username == "cmdb_agent" or g.user.uid == rule['uid']):
rule['extra_option'].pop('secret', None)
else:
rule['extra_option']['secret'] = AESCrypto.decrypt(rule['extra_option']['secret'])
if oneagent_id and rule['agent_id'] == oneagent_id:
result.append(rule)
elif rule['query_expr']:
query = rule['query_expr'].lstrip('q').lstrip('=')
s = search(query, fl=['_id'], count=1000000)
try:
response, _, _, _, _, _ = s.search()
except SearchError as e:
return abort(400, str(e))
for i in (response or []):
if i.get('_id') == ci_id:
result.append(rule)
break
elif not rule['agent_id'] and not rule['query_expr'] and rule['adr_id']:
adr = AutoDiscoveryRuleCRUD.get_by_id(rule['adr_id'])
if not adr:
continue
if adr.type in (AutoDiscoveryType.SNMP, AutoDiscoveryType.HTTP):
continue
if not rule['updated_at']:
continue
result.append(rule)
new_last_update_at = ""
for i in result:
i['adr'] = AutoDiscoveryRule.get_by_id(i['adr_id']).to_dict()
__last_update_at = max([i['updated_at'] or "", i['created_at'] or "",
i['adr']['created_at'] or "", i['adr']['updated_at'] or ""])
if new_last_update_at < __last_update_at:
new_last_update_at = __last_update_at
if not last_update_at or new_last_update_at > last_update_at:
return result, new_last_update_at
else:
return [], new_last_update_at
@staticmethod
def __valid_exec_target(agent_id, query_expr):
_is_app_admin = is_app_admin("cmdb")
if not agent_id and not query_expr and not _is_app_admin:
return abort(403, ErrFormat.adt_target_all_no_permission)
if _is_app_admin:
return
if agent_id and isinstance(agent_id, str) and agent_id.startswith("0x"):
agent_id = agent_id.strip()
q = "op_duty:{0},-rd_duty:{0},oneagent_id:{1}"
s = search(q.format(g.user.username, agent_id.strip()))
try:
response, _, _, _, _, _ = s.search()
if response:
return
except SearchError as e:
current_app.logger.warning(e)
return abort(400, str(e))
s = search(q.format(g.user.nickname, agent_id.strip()))
try:
response, _, _, _, _, _ = s.search()
if response:
return
except SearchError as e:
current_app.logger.warning(e)
return abort(400, str(e))
if query_expr.strip():
query_expr = query_expr.strip()
if query_expr.startswith('q='):
query_expr = query_expr[2:]
s = search(query_expr, count=1000000)
try:
response, _, _, _, _, _ = s.search()
for i in response:
if g.user.username not in (i.get('rd_duty') or []) and g.user.username not in \
(i.get('op_duty') or []) and g.user.nickname not in (i.get('rd_duty') or []) and \
g.user.nickname not in (i.get('op_duty') or []):
return abort(403, ErrFormat.adt_target_expr_no_permission.format(
i.get("{}_name".format(i.get('ci_type')))))
except SearchError as e:
current_app.logger.warning(e)
return abort(400, str(e))
def _can_add(self, **kwargs):
self.cls.get_by(type_id=kwargs['type_id'], adr_id=kwargs.get('adr_id') or None) and abort(
400, ErrFormat.ad_duplicate)
# self.__valid_exec_target(kwargs.get('agent_id'), kwargs.get('query_expr'))
if kwargs.get('adr_id'):
adr = AutoDiscoveryRule.get_by_id(kwargs['adr_id']) or abort(
404, ErrFormat.adr_not_found.format("id={}".format(kwargs['adr_id'])))
if not adr.is_plugin:
other = self.cls.get_by(adr_id=adr.id, first=True, to_dict=False)
if other:
ci_type = CITypeCache.get(other.type_id)
return abort(400, ErrFormat.adr_default_ref_once.format(ci_type.alias))
if kwargs.get('is_plugin') and kwargs.get('plugin_script'):
kwargs = check_plugin_script(**kwargs)
if isinstance(kwargs.get('extra_option'), dict) and kwargs['extra_option'].get('secret'):
kwargs['extra_option']['secret'] = AESCrypto.encrypt(kwargs['extra_option']['secret'])
kwargs['uid'] = g.user.uid
return kwargs
def _can_update(self, **kwargs):
existed = self.cls.get_by_id(kwargs['_id']) or abort(
404, ErrFormat.ad_not_found.format("id={}".format(kwargs['_id'])))
self.__valid_exec_target(kwargs.get('agent_id'), kwargs.get('query_expr'))
if isinstance(kwargs.get('extra_option'), dict) and kwargs['extra_option'].get('secret'):
if g.user.uid != existed.uid:
return abort(403, ErrFormat.adt_secret_no_permission)
return existed
def update(self, _id, **kwargs):
if kwargs.get('is_plugin') and kwargs.get('plugin_script'):
kwargs = check_plugin_script(**kwargs)
if isinstance(kwargs.get('extra_option'), dict) and kwargs['extra_option'].get('secret'):
kwargs['extra_option']['secret'] = AESCrypto.encrypt(kwargs['extra_option']['secret'])
return super(AutoDiscoveryCITypeCRUD, self).update(_id, filter_none=False, **kwargs)
def _can_delete(self, **kwargs):
if AutoDiscoveryCICRUD.get_by_adt_id(kwargs['_id']):
return abort(400, ErrFormat.cannot_delete_adt)
existed = self.cls.get_by_id(kwargs['_id']) or abort(
404, ErrFormat.ad_not_found.format("id={}".format(kwargs['_id'])))
return existed
class AutoDiscoveryCICRUD(DBMixin):
cls = AutoDiscoveryCI
@classmethod
def get_by_adt_id(cls, adt_id):
return cls.cls.get_by(adt_id=adt_id, to_dict=False)
@classmethod
def get_type_name(cls, adc_id):
adc = cls.cls.get_by_id(adc_id) or abort(404, ErrFormat.adc_not_found)
ci_type = CITypeCache.get(adc.type_id)
return ci_type and ci_type.name
@staticmethod
def get_ci_types(need_other):
result = CITypeGroupManager.get(need_other, False)
adt = {i.type_id for i in AutoDiscoveryCITypeCRUD.get_all()}
for item in result:
item['ci_types'] = [i for i in (item.get('ci_types') or []) if i['id'] in adt]
return result
@staticmethod
def get_attributes_by_type_id(type_id):
from api.lib.cmdb.cache import CITypeAttributesCache
attributes = [i[1] for i in CITypeAttributesCache.get2(type_id) or []]
attr_names = set()
adts = AutoDiscoveryCITypeCRUD.get_by_type_id(type_id)
for adt in adts:
attr_names |= set((adt.attributes or {}).values())
return [attr.to_dict() for attr in attributes if attr.name in attr_names]
@classmethod
def search(cls, page, page_size, fl=None, **kwargs):
type_id = kwargs['type_id']
adts = AutoDiscoveryCITypeCRUD.get_by_type_id(type_id)
if not adts:
return 0, []
adt2attr_map = {i.id: i.attributes or {} for i in adts}
query = db.session.query(cls.cls).filter(cls.cls.deleted.is_(False))
count_query = db.session.query(func.count(cls.cls.id)).filter(cls.cls.deleted.is_(False))
for k in kwargs:
if hasattr(cls.cls, k):
query = query.filter(getattr(cls.cls, k) == kwargs[k])
count_query = count_query.filter(getattr(cls.cls, k) == kwargs[k])
query = query.order_by(cls.cls.is_accept.desc()).order_by(cls.cls.id.desc())
result = []
for i in query.offset((page - 1) * page_size).limit(page_size):
item = i.to_dict()
adt_id = item['adt_id']
item['instance'] = {adt2attr_map[adt_id][k]: v for k, v in item.get('instance').items()
if (not fl or k in fl) and adt2attr_map.get(adt_id, {}).get(k)}
result.append(item)
numfound = query.count()
return numfound, result
@staticmethod
def _get_unique_key(type_id):
ci_type = CITypeCache.get(type_id)
if ci_type:
attr = CITypeAttributeCache.get(type_id, ci_type.unique_id)
return attr and attr.name
def _can_add(self, **kwargs):
pass
def upsert(self, **kwargs):
adt = AutoDiscoveryCIType.get_by_id(kwargs['adt_id']) or abort(404, ErrFormat.adt_not_found)
existed = self.cls.get_by(type_id=kwargs['type_id'],
unique_value=kwargs.get("unique_value"),
first=True, to_dict=False)
changed = False
if existed is not None:
if existed.instance != kwargs['instance']:
existed.update(filter_none=False, **kwargs)
changed = True
else:
existed = self.cls.create(**kwargs)
changed = True
if adt.auto_accept and changed:
try:
self.accept(existed)
except Exception as e:
return abort(400, str(e))
elif changed:
existed.update(is_accept=False, accept_time=None, accept_by=None, filter_none=False)
return existed
def _can_update(self, **kwargs):
existed = self.cls.get_by_id(kwargs['_id']) or abort(404, ErrFormat.adc_not_found)
return existed
def _can_delete(self, **kwargs):
return self._can_update(**kwargs)
def delete(self, _id):
inst = self._can_delete(_id=_id)
inst.delete()
self._after_delete(inst)
return inst
@classmethod
def delete2(cls, type_id, unique_value):
existed = cls.cls.get_by(type_id=type_id, unique_value=unique_value, first=True, to_dict=False) or abort(
404, ErrFormat.adc_not_found)
if current_app.config.get("USE_ACL"):
ci_type = CITypeCache.get(type_id) or abort(404, ErrFormat.ci_type_not_found)
not is_app_admin("cmdb") and validate_permission(ci_type.name, ResourceTypeEnum.CI, PermEnum.DELETE, "cmdb")
existed.delete()
# TODO: delete ci
@classmethod
def accept(cls, adc, adc_id=None, nickname=None):
if adc_id is not None:
adc = cls.cls.get_by_id(adc_id) or abort(404, ErrFormat.adc_not_found)
adt = AutoDiscoveryCITypeCRUD.get_by_id(adc.adt_id) or abort(404, ErrFormat.adt_not_found)
ci_id = None
if adt.attributes:
ci_dict = {adt.attributes[k]: v for k, v in adc.instance.items() if k in adt.attributes}
ci_id = CIManager.add(adc.type_id, is_auto_discovery=True, **ci_dict)
relation_adts = AutoDiscoveryCIType.get_by(type_id=adt.type_id, adr_id=None, to_dict=False)
for r_adt in relation_adts:
if r_adt.relation and ci_id is not None:
ad_key, cmdb_key = None, {}
for ad_key in r_adt.relation:
cmdb_key = r_adt.relation[ad_key]
query = "_type:{},{}:{}".format(cmdb_key.get('type_name'), cmdb_key.get('attr_name'),
adc.instance.get(ad_key))
s = search(query)
try:
response, _, _, _, _, _ = s.search()
except SearchError as e:
current_app.logger.warning(e)
return abort(400, str(e))
relation_ci_id = response and response[0]['_id']
if relation_ci_id:
try:
CIRelationManager.add(ci_id, relation_ci_id)
except:
try:
CIRelationManager.add(relation_ci_id, ci_id)
except:
pass
adc.update(is_accept=True, accept_by=nickname or g.user.nickname, accept_time=datetime.datetime.now())
class AutoDiscoveryHTTPManager(object):
@staticmethod
def get_categories(name):
return (ClOUD_MAP.get(name) or {}).get('categories') or []
@staticmethod
def get_attributes(name, category):
tpt = ((ClOUD_MAP.get(name) or {}).get('map') or {}).get(category)
if tpt and os.path.exists(os.path.join(PWD, tpt)):
with open(os.path.join(PWD, tpt)) as f:
return json.loads(f.read())
return []
class AutoDiscoverySNMPManager(object):
@staticmethod
def get_attributes():
if os.path.exists(os.path.join(PWD, "templates/net_device.json")):
with open(os.path.join(PWD, "templates/net_device.json")) as f:
return json.loads(f.read())
return []

53
cmdb-api/api/lib/cmdb/auto_discovery/const.py Normal file
View File

@ -0,0 +1,53 @@
# -*- coding:utf-8 -*-
from api.lib.cmdb.const import AutoDiscoveryType
DEFAULT_HTTP = [
dict(name="阿里云", type=AutoDiscoveryType.HTTP, is_inner=True, is_plugin=False,
option={'icon': {'name': 'caise-aliyun'}}),
dict(name="腾讯云", type=AutoDiscoveryType.HTTP, is_inner=True, is_plugin=False,
option={'icon': {'name': 'caise-tengxunyun'}}),
dict(name="华为云", type=AutoDiscoveryType.HTTP, is_inner=True, is_plugin=False,
option={'icon': {'name': 'caise-huaweiyun'}}),
dict(name="AWS", type=AutoDiscoveryType.HTTP, is_inner=True, is_plugin=False,
option={'icon': {'name': 'caise-aws'}}),
dict(name="交换机", type=AutoDiscoveryType.SNMP, is_inner=True, is_plugin=False,
option={'icon': {'name': 'caise-jiaohuanji'}}),
dict(name="路由器", type=AutoDiscoveryType.SNMP, is_inner=True, is_plugin=False,
option={'icon': {'name': 'caise-luyouqi'}}),
dict(name="防火墙", type=AutoDiscoveryType.SNMP, is_inner=True, is_plugin=False,
option={'icon': {'name': 'caise-fanghuoqiang'}}),
dict(name="打印机", type=AutoDiscoveryType.SNMP, is_inner=True, is_plugin=False,
option={'icon': {'name': 'caise-dayinji'}}),
]
ClOUD_MAP = {
"aliyun": {
"categories": ["云服务器 ECS"],
"map": {
"云服务器 ECS": "templates/aliyun_ecs.json",
}
},
"tencentcloud": {
"categories": ["云服务器 CVM"],
"map": {
"云服务器 CVM": "templates/tencent_cvm.json",
}
},
"huaweicloud": {
"categories": ["云服务器 ECS"],
"map": {
"云服务器 ECS": "templates/huaweicloud_ecs.json",
}
},
"aws": {
"categories": ["云服务器 EC2"],
"map": {
"云服务器 EC2": "templates/aws_ec2.json",
}
},
}

647
cmdb-api/api/lib/cmdb/auto_discovery/templates/aliyun_ecs.json Normal file
View File

@ -0,0 +1,647 @@
[
{
"name": "CreationTime",
"type": "文本",
"example": "2017-12-10T04:04Z",
"desc": "\u5b9e\u4f8b\u521b\u5efa\u65f6\u95f4\u3002\u4ee5ISO 8601\u4e3a\u6807\u51c6\uff0c\u5e76\u4f7f\u7528UTC+0\u65f6\u95f4\uff0c\u683c\u5f0f\u4e3ayyyy-MM-ddTHH:mmZ\u3002\u66f4\u591a\u4fe1\u606f\uff0c\u8bf7\u53c2\u89c1[ISO 8601](~~25696~~)\u3002"
},
{
"name": "SerialNumber",
"type": "文本",
"example": "51d1353b-22bf-4567-a176-8b3e12e4****",
"desc": "\u5b9e\u4f8b\u5e8f\u5217\u53f7\u3002"
},
{
"name": "Status",
"type": "文本",
"example": "Running",
"desc": "\u5b9e\u4f8b\u72b6\u6001\u3002"
},
{
"name": "DeploymentSetId",
"type": "文本",
"example": "ds-bp67acfmxazb4p****",
"desc": "\u90e8\u7f72\u96c6ID\u3002"
},
{
"name": "KeyPairName",
"type": "文本",
"example": "testKeyPairName",
"desc": "\u5bc6\u94a5\u5bf9\u540d\u79f0\u3002"
},
{
"name": "SaleCycle",
"type": "文本",
"example": "month",
"desc": "> \u8be5\u53c2\u6570\u5df2\u5f03\u7528\uff0c\u4e0d\u518d\u8fd4\u56de\u6709\u610f\u4e49\u7684\u6570\u636e\u3002"
},
{
"name": "SpotStrategy",
"type": "文本",
"example": "NoSpot",
"desc": "\u6309\u91cf\u5b9e\u4f8b\u7684\u7ade\u4ef7\u7b56\u7565\u3002\u53ef\u80fd\u503c\uff1a\n\n- NoSpot\uff1a\u6b63\u5e38\u6309\u91cf\u4ed8\u8d39\u5b9e\u4f8b\u3002\n- SpotWithPriceLimit\uff1a\u8bbe\u7f6e\u4e0a\u9650\u4ef7\u683c\u7684\u62a2\u5360\u5f0f\u5b9e\u4f8b\u3002\n- SpotAsPriceGo\uff1a\u7cfb\u7edf\u81ea\u52a8\u51fa\u4ef7\uff0c\u6700\u9ad8\u6309\u91cf\u4ed8\u8d39\u4ef7\u683c\u7684\u62a2\u5360\u5f0f\u5b9e\u4f8b\u3002"
},
{
"name": "DeviceAvailable",
"type": "boolean",
"example": "true",
"desc": "\u5b9e\u4f8b\u662f\u5426\u53ef\u4ee5\u6302\u8f7d\u6570\u636e\u76d8\u3002"
},
{
"name": "LocalStorageCapacity",
"type": "整数",
"example": "1000",
"desc": "\u5b9e\u4f8b\u6302\u8f7d\u7684\u672c\u5730\u5b58\u50a8\u5bb9\u91cf\u3002"
},
{
"name": "Description",
"type": "文本",
"example": "testDescription",
"desc": "\u5b9e\u4f8b\u63cf\u8ff0\u3002"
},
{
"name": "SpotDuration",
"type": "整数",
"example": "1",
"desc": "\u62a2\u5360\u5f0f\u5b9e\u4f8b\u7684\u4fdd\u7559\u65f6\u957f\uff0c\u5355\u4f4d\u4e3a\u5c0f\u65f6\u3002\u53ef\u80fd\u503c\u4e3a0~6\u3002\n\n- \u4fdd\u7559\u65f6\u957f2~6\u6b63\u5728\u9080\u6d4b\u4e2d\uff0c\u5982\u9700\u5f00\u901a\u8bf7\u63d0\u4ea4\u5de5\u5355\u3002\n- \u503c\u4e3a0\uff0c\u5219\u4e3a\u65e0\u4fdd\u62a4\u671f\u6a21\u5f0f\u3002\n\n>\u5f53SpotStrategy\u503c\u4e3aSpotWithPriceLimit\u6216SpotAsPriceGo\u65f6\u8fd4\u56de\u8be5\u53c2\u6570\u3002"
},
{
"name": "InstanceNetworkType",
"type": "文本",
"example": "vpc",
"desc": "\u5b9e\u4f8b\u7f51\u7edc\u7c7b\u578b\u3002\u53ef\u80fd\u503c\uff1a\n\n- classic\uff1a\u7ecf\u5178\u7f51\u7edc\u3002\n- vpc\uff1a\u4e13\u6709\u7f51\u7edcVPC\u3002"
},
{
"name": "InstanceName",
"type": "文本",
"example": "InstanceNameTest",
"desc": "\u5b9e\u4f8b\u540d\u79f0\u3002"
},
{
"name": "OSNameEn",
"type": "文本",
"example": "CentOS 7.4 64 bit",
"desc": "\u5b9e\u4f8b\u64cd\u4f5c\u7cfb\u7edf\u7684\u82f1\u6587\u540d\u79f0\u3002"
},
{
"name": "HpcClusterId",
"type": "文本",
"example": "hpc-bp67acfmxazb4p****",
"desc": "\u5b9e\u4f8b\u6240\u5c5e\u7684HPC\u96c6\u7fa4ID\u3002"
},
{
"name": "SpotPriceLimit",
"type": "float",
"example": "0.98",
"desc": "\u5b9e\u4f8b\u7684\u6bcf\u5c0f\u65f6\u6700\u9ad8\u4ef7\u683c\u3002\u652f\u6301\u6700\u59273\u4f4d\u5c0f\u6570\uff0c\u53c2\u6570SpotStrategy=SpotWithPriceLimit\u65f6\uff0c\u8be5\u53c2\u6570\u751f\u6548\u3002"
},
{
"name": "Memory",
"type": "整数",
"example": "16384",
"desc": "\u5185\u5b58\u5927\u5c0f\uff0c\u5355\u4f4d\u4e3aMiB\u3002"
},
{
"name": "OSName",
"type": "文本",
"example": "CentOS 7.4 64 \u4f4d",
"desc": "\u5b9e\u4f8b\u7684\u64cd\u4f5c\u7cfb\u7edf\u540d\u79f0\u3002"
},
{
"name": "DeploymentSetGroupNo",
"type": "整数",
"example": "1",
"desc": "ECS\u5b9e\u4f8b\u7ed1\u5b9a\u90e8\u7f72\u96c6\u5206\u6563\u90e8\u7f72\u65f6\uff0c\u5b9e\u4f8b\u5728\u90e8\u7f72\u96c6\u4e2d\u7684\u5206\u7ec4\u4f4d\u7f6e\u3002"
},
{
"name": "ImageId",
"type": "文本",
"example": "m-bp67acfmxazb4p****",
"desc": "\u5b9e\u4f8b\u8fd0\u884c\u7684\u955c\u50cfID\u3002"
},
{
"name": "VlanId",
"type": "文本",
"example": "10",
"desc": "\u5b9e\u4f8b\u7684VLAN ID\u3002\n\n>\u8be5\u53c2\u6570\u5373\u5c06\u88ab\u5f03\u7528\uff0c\u4e3a\u63d0\u9ad8\u517c\u5bb9\u6027\uff0c\u8bf7\u5c3d\u91cf\u4f7f\u7528\u5176\u4ed6\u53c2\u6570\u3002"
},
{
"name": "ClusterId",
"type": "文本",
"example": "c-bp67acfmxazb4p****",
"desc": "\u5b9e\u4f8b\u6240\u5728\u7684\u96c6\u7fa4ID\u3002\n\n>\u8be5\u53c2\u6570\u5373\u5c06\u88ab\u5f03\u7528\uff0c\u4e3a\u63d0\u9ad8\u517c\u5bb9\u6027\uff0c\u8bf7\u5c3d\u91cf\u4f7f\u7528\u5176\u4ed6\u53c2\u6570\u3002"
},
{
"name": "GPUSpec",
"type": "文本",
"example": "NVIDIA V100",
"desc": "\u5b9e\u4f8b\u89c4\u683c\u9644\u5e26\u7684GPU\u7c7b\u578b\u3002"
},
{
"name": "AutoReleaseTime",
"type": "文本",
"example": "2017-12-10T04:04Z",
"desc": "\u6309\u91cf\u4ed8\u8d39\u5b9e\u4f8b\u7684\u81ea\u52a8\u91ca\u653e\u65f6\u95f4\u3002"
},
{
"name": "DeletionProtection",
"type": "boolean",
"example": "false",
"desc": "\u5b9e\u4f8b\u91ca\u653e\u4fdd\u62a4\u5c5e\u6027\uff0c\u6307\u5b9a\u662f\u5426\u652f\u6301\u901a\u8fc7\u63a7\u5236\u53f0\u6216API\uff08DeleteInstance\uff09\u91ca\u653e\u5b9e\u4f8b\u3002\n\n- true\uff1a\u5df2\u5f00\u542f\u5b9e\u4f8b\u91ca\u653e\u4fdd\u62a4\u3002\n- false\uff1a\u672a\u5f00\u542f\u5b9e\u4f8b\u91ca\u653e\u4fdd\u62a4\u3002\n\n> \u8be5\u5c5e\u6027\u4ec5\u9002\u7528\u4e8e\u6309\u91cf\u4ed8\u8d39\u5b9e\u4f8b\uff0c\u4e14\u53ea\u80fd\u9650\u5236\u624b\u52a8\u91ca\u653e\u64cd\u4f5c\uff0c\u5bf9\u7cfb\u7edf\u91ca\u653e\u64cd\u4f5c\u4e0d\u751f\u6548\u3002"
},
{
"name": "StoppedMode",
"type": "文本",
"example": "KeepCharging",
"desc": "\u5b9e\u4f8b\u505c\u673a\u540e\u662f\u5426\u7ee7\u7eed\u6536\u8d39\u3002\u53ef\u80fd\u503c\uff1a\n\n- KeepCharging\uff1a\u505c\u673a\u540e\u7ee7\u7eed\u6536\u8d39\uff0c\u4e3a\u60a8\u7ee7\u7eed\u4fdd\u7559\u5e93\u5b58\u8d44\u6e90\u3002\n- StopCharging\uff1a\u505c\u673a\u540e\u4e0d\u6536\u8d39\u3002\u505c\u673a\u540e\uff0c\u6211\u4eec\u91ca\u653e\u5b9e\u4f8b\u5bf9\u5e94\u7684\u8d44\u6e90\uff0c\u4f8b\u5982vCPU\u3001\u5185\u5b58\u548c\u516c\u7f51IP\u7b49\u8d44\u6e90\u3002\u91cd\u542f\u662f\u5426\u6210\u529f\u4f9d\u8d56\u4e8e\u5f53\u524d\u5730\u57df\u4e2d\u662f\u5426\u4ecd\u6709\u8d44\u6e90\u5e93\u5b58\u3002\n- Not-applicable\uff1a\u672c\u5b9e\u4f8b\u4e0d\u652f\u6301\u505c\u673a\u4e0d\u6536\u8d39\u529f\u80fd\u3002"
},
{
"name": "GPUAmount",
"type": "整数",
"example": "4",
"desc": "\u5b9e\u4f8b\u89c4\u683c\u9644\u5e26\u7684GPU\u6570\u91cf\u3002"
},
{
"name": "HostName",
"type": "文本",
"example": "testHostName",
"desc": "\u5b9e\u4f8b\u4e3b\u673a\u540d\u3002"
},
{
"name": "InstanceId",
"type": "文本",
"example": "i-bp67acfmxazb4p****",
"desc": "\u5b9e\u4f8bID\u3002"
},
{
"name": "InternetMaxBandwidthOut",
"type": "整数",
"example": "5",
"desc": "\u516c\u7f51\u51fa\u5e26\u5bbd\u6700\u5927\u503c\uff0c\u5355\u4f4d\u4e3aMbit/s\u3002"
},
{
"name": "InternetMaxBandwidthIn",
"type": "整数",
"example": "50",
"desc": "\u516c\u7f51\u5165\u5e26\u5bbd\u6700\u5927\u503c\uff0c\u5355\u4f4d\u4e3aMbit/s\u3002"
},
{
"name": "InstanceType",
"type": "文本",
"example": "ecs.g5.large",
"desc": "\u5b9e\u4f8b\u89c4\u683c\u3002"
},
{
"name": "InstanceChargeType",
"type": "文本",
"example": "PostPaid",
"desc": "\u5b9e\u4f8b\u7684\u8ba1\u8d39\u65b9\u5f0f\u3002\u53ef\u80fd\u503c\uff1a\n\n- PrePaid\uff1a\u5305\u5e74\u5305\u6708\u3002\n- PostPaid\uff1a\u6309\u91cf\u4ed8\u8d39\u3002"
},
{
"name": "RegionId",
"type": "文本",
"example": "cn-hangzhou",
"desc": "\u5b9e\u4f8b\u6240\u5c5e\u5730\u57dfID\u3002"
},
{
"name": "IoOptimized",
"type": "boolean",
"example": "true",
"desc": "\u662f\u5426\u4e3aI/O\u4f18\u5316\u578b\u5b9e\u4f8b\u3002"
},
{
"name": "StartTime",
"type": "文本",
"example": "2017-12-10T04:04Z",
"desc": "\u5b9e\u4f8b\u6700\u8fd1\u4e00\u6b21\u7684\u542f\u52a8\u65f6\u95f4\u3002\u4ee5ISO8601\u4e3a\u6807\u51c6\uff0c\u5e76\u4f7f\u7528UTC+0\u65f6\u95f4\uff0c\u683c\u5f0f\u4e3ayyyy-MM-ddTHH:mmZ\u3002\u66f4\u591a\u4fe1\u606f\uff0c\u8bf7\u53c2\u89c1[ISO8601](~~25696~~)\u3002"
},
{
"name": "Cpu",
"type": "整数",
"example": "8",
"desc": "vCPU\u6570\u3002"
},
{
"name": "LocalStorageAmount",
"type": "整数",
"example": "2",
"desc": "\u5b9e\u4f8b\u6302\u8f7d\u7684\u672c\u5730\u5b58\u50a8\u6570\u91cf\u3002"
},
{
"name": "ExpiredTime",
"type": "文本",
"example": "2017-12-10T04:04Z",
"desc": "\u8fc7\u671f\u65f6\u95f4\u3002\u4ee5ISO8601\u4e3a\u6807\u51c6\uff0c\u5e76\u4f7f\u7528UTC+0\u65f6\u95f4\uff0c\u683c\u5f0f\u4e3ayyyy-MM-ddTHH:mmZ\u3002\u66f4\u591a\u4fe1\u606f\uff0c\u8bf7\u53c2\u89c1[ISO8601](~~25696~~)\u3002"
},
{
"name": "ResourceGroupId",
"type": "文本",
"example": "rg-bp67acfmxazb4p****",
"desc": "\u5b9e\u4f8b\u6240\u5c5e\u7684\u4f01\u4e1a\u8d44\u6e90\u7ec4ID\u3002"
},
{
"name": "InternetChargeType",
"type": "文本",
"example": "PayByTraffic",
"desc": "\u7f51\u7edc\u8ba1\u8d39\u7c7b\u578b\u3002\u53ef\u80fd\u503c\uff1a\n\n- PayByBandwidth\uff1a\u6309\u56fa\u5b9a\u5e26\u5bbd\u8ba1\u8d39\u3002\n- PayByTraffic\uff1a\u6309\u4f7f\u7528\u6d41\u91cf\u8ba1\u8d39\u3002"
},
{
"name": "ZoneId",
"type": "文本",
"example": "cn-hangzhou-g",
"desc": "\u5b9e\u4f8b\u6240\u5c5e\u53ef\u7528\u533a\u3002"
},
{
"name": "Recyclable",
"type": "boolean",
"example": "false",
"desc": "\u5b9e\u4f8b\u662f\u5426\u53ef\u4ee5\u56de\u6536\u3002"
},
{
"name": "ISP",
"type": "文本",
"example": "null",
"desc": "> \u8be5\u53c2\u6570\u6b63\u5728\u9080\u6d4b\u4e2d\uff0c\u6682\u672a\u5f00\u653e\u4f7f\u7528\u3002"
},
{
"name": "CreditSpecification",
"type": "文本",
"example": "Standard",
"desc": "\u4fee\u6539\u7a81\u53d1\u6027\u80fd\u5b9e\u4f8b\u7684\u8fd0\u884c\u6a21\u5f0f\u3002\u53ef\u80fd\u503c\uff1a\n\n- Standard\uff1a\u6807\u51c6\u6a21\u5f0f\u3002\u6709\u5173\u5b9e\u4f8b\u6027\u80fd\u7684\u66f4\u591a\u4fe1\u606f\uff0c\u8bf7\u53c2\u89c1[\u4ec0\u4e48\u662f\u7a81\u53d1\u6027\u80fd\u5b9e\u4f8b](~~59977~~)\u4e2d\u7684\u6027\u80fd\u7ea6\u675f\u6a21\u5f0f\u7ae0\u8282\u3002\n- Unlimited\uff1a\u65e0\u6027\u80fd\u7ea6\u675f\u6a21\u5f0f\uff0c\u6709\u5173\u5b9e\u4f8b\u6027\u80fd\u7684\u66f4\u591a\u4fe1\u606f\uff0c\u8bf7\u53c2\u89c1[\u4ec0\u4e48\u662f\u7a81\u53d1\u6027\u80fd\u5b9e\u4f8b](~~59977~~)\u4e2d\u7684\u65e0\u6027\u80fd\u7ea6\u675f\u6a21\u5f0f\u7ae0\u8282\u3002"
},
{
"name": "InstanceTypeFamily",
"type": "文本",
"example": "ecs.g5",
"desc": "\u5b9e\u4f8b\u89c4\u683c\u65cf\u3002"
},
{
"name": "OSType",
"type": "文本",
"example": "linux",
"desc": "\u5b9e\u4f8b\u7684\u64cd\u4f5c\u7cfb\u7edf\u7c7b\u578b\uff0c\u5206\u4e3aWindows Server\u548cLinux\u4e24\u79cd\u3002\u53ef\u80fd\u503c\uff1a\n\n- windows\u3002\n- linux\u3002"
},
{
"name": "NetworkInterfaces",
"type": "json",
"example": {
"type": "json",
"properties": {
"Type": {
"description": "\u5f39\u6027\u7f51\u5361\u7c7b\u578b\u3002\u53ef\u80fd\u503c\uff1a\n- Primary\uff1a\u4e3b\u7f51\u5361\u3002\n- Secondary\uff1a\u8f85\u52a9\u5f39\u6027\u7f51\u5361\u3002",
"type": "文本",
"example": "Primary"
},
"MacAddress": {
"description": "\u5f39\u6027\u7f51\u5361\u7684MAC\u5730\u5740\u3002",
"type": "文本",
"example": "00:16:3e:32:b4:**"
},
"PrimaryIpAddress": {
"description": "\u5f39\u6027\u7f51\u5361\u4e3b\u79c1\u6709IP\u5730\u5740\u3002",
"type": "文本",
"example": "172.17.**.***"
},
"NetworkInterfaceId": {
"description": "\u5f39\u6027\u7f51\u5361\u7684ID\u3002",
"type": "文本",
"example": "eni-2zeh9atclduxvf1z****"
},
"PrivateIpSets": {
"type": "array",
"items": {
"type": "json",
"properties": {
"PrivateIpAddress": {
"description": "\u5b9e\u4f8b\u7684\u79c1\u7f51IP\u5730\u5740\u3002",
"type": "文本",
"example": "172.17.**.**"
},
"Primary": {
"description": "\u662f\u5426\u662f\u4e3b\u79c1\u7f51IP\u5730\u5740\u3002",
"type": "boolean",
"example": "true"
}
}
},
"description": "PrivateIpSet\u7ec4\u6210\u7684\u96c6\u5408\u3002"
},
"Ipv6Sets": {
"type": "array",
"items": {
"type": "json",
"properties": {
"Ipv6Address": {
"description": "\u4e3a\u5f39\u6027\u7f51\u5361\u6307\u5b9a\u7684IPv6\u5730\u5740\u3002",
"type": "文本",
"example": "2408:4321:180:1701:94c7:bc38:3bfa:***"
}
}
},
"description": "\u4e3a\u5f39\u6027\u7f51\u5361\u5206\u914d\u7684IPv6\u5730\u5740\u96c6\u5408\u3002\u4ec5\u5f53\u8bf7\u6c42\u53c2\u6570`AdditionalAttributes.N`\u53d6\u503c\u4e3a`NETWORK_PRIMARY_ENI_IP`\u65f6\uff0c\u624d\u4f1a\u8fd4\u56de\u8be5\u53c2\u6570\u503c\u3002"
},
"Ipv4PrefixSets": {
"type": "array",
"items": {
"type": "json",
"properties": {
"Ipv4Prefix": {
"description": "IPv4\u524d\u7f00\u3002",
"type": "文本",
"example": "47.122.*.*/19"
}
}
},
"description": "IPv4\u524d\u7f00\u96c6\u5408\u3002"
},
"Ipv6PrefixSets": {
"type": "array",
"items": {
"type": "json",
"properties": {
"Ipv6Prefix": {
"description": "IPv6\u524d\u7f00\u3002",
"type": "文本",
"example": "2001:1111:*:*::/64"
}
}
},
"description": "IPv6\u524d\u7f00\u96c6\u5408\u3002"
}
},
"description": "\u5b9e\u4f8b\u5305\u542b\u7684\u5f39\u6027\u7f51\u5361\u96c6\u5408\u3002"
},
"desc": "\u5b9e\u4f8b\u5305\u542b\u7684\u5f39\u6027\u7f51\u5361\u96c6\u5408\u3002"
},
{
"name": "OperationLocks",
"type": "文本、多值",
"example": {
"type": "json",
"properties": {
"LockMsg": {
"description": "\u5b9e\u4f8b\u88ab\u9501\u5b9a\u7684\u63cf\u8ff0\u4fe1\u606f\u3002",
"type": "文本",
"example": "The specified instance is locked due to financial reason."
},
"LockReason": {
"description": "\u9501\u5b9a\u7c7b\u578b\u3002\u53ef\u80fd\u503c\uff1a\n\n- financial\uff1a\u56e0\u6b20\u8d39\u88ab\u9501\u5b9a\u3002\n- security\uff1a\u56e0\u5b89\u5168\u539f\u56e0\u88ab\u9501\u5b9a\u3002\n- Recycling\uff1a\u62a2\u5360\u5f0f\u5b9e\u4f8b\u7684\u5f85\u91ca\u653e\u9501\u5b9a\u72b6\u6001\u3002\n- dedicatedhostfinancial\uff1a\u56e0\u4e3a\u4e13\u6709\u5bbf\u4e3b\u673a\u6b20\u8d39\u5bfc\u81f4ECS\u5b9e\u4f8b\u88ab\u9501\u5b9a\u3002\n- refunded\uff1a\u56e0\u9000\u6b3e\u88ab\u9501\u5b9a\u3002",
"type": "文本",
"example": "Recycling"
}
}
},
"desc": "\u5b9e\u4f8b\u7684\u9501\u5b9a\u539f\u56e0\u3002"
},
{
"name": "Tags",
"type": "json",
"example": {
"type": "json",
"properties": {
"TagValue": {
"description": "\u5b9e\u4f8b\u7684\u6807\u7b7e\u503c\u3002",
"type": "文本",
"example": "TestValue"
},
"TagKey": {
"description": "\u5b9e\u4f8b\u7684\u6807\u7b7e\u952e\u3002",
"type": "文本",
"example": "TestKey"
}
}
},
"desc": "\u5b9e\u4f8b\u7684\u6807\u7b7e\u96c6\u5408\u3002"
},
{
"name": "RdmaIpAddress",
"type": "文本、多值",
"example": {
"description": "HPC\u5b9e\u4f8b\u7684Rdma\u7f51\u7edcIP\u3002",
"type": "文本",
"example": "10.10.10.102"
},
"desc": "HPC\u5b9e\u4f8b\u7684Rdma\u7f51\u7edcIP\u5217\u8868\u3002"
},
{
"name": "SecurityGroupIds",
"type": "文本、多值",
"example": {
"description": "\u5b89\u5168\u7ec4ID\u3002",
"type": "文本",
"example": "sg-bp67acfmxazb4p****"
},
"desc": "\u5b9e\u4f8b\u6240\u5c5e\u5b89\u5168\u7ec4ID\u5217\u8868\u3002"
},
{
"name": "PublicIpAddress",
"type": "文本、多值",
"example": {
"description": "\u5b9e\u4f8b\u516c\u7f51IP\u5730\u5740\u3002",
"type": "文本",
"example": "121.40.**.**"
},
"desc": "\u5b9e\u4f8b\u516c\u7f51IP\u5730\u5740\u5217\u8868\u3002"
},
{
"name": "InnerIpAddress",
"type": "文本、多值",
"example": {
"description": "\u7ecf\u5178\u7f51\u7edc\u7c7b\u578b\u5b9e\u4f8b\u7684\u5185\u7f51IP\u5730\u5740\u3002",
"type": "文本",
"example": "10.170.**.**"
},
"desc": "\u7ecf\u5178\u7f51\u7edc\u7c7b\u578b\u5b9e\u4f8b\u7684\u5185\u7f51IP\u5730\u5740\u5217\u8868\u3002"
},
{
"name": "VpcAttributes",
"type": "json",
"example": {
"VpcId": {
"description": "\u4e13\u6709\u7f51\u7edcVPC ID\u3002",
"type": "文本",
"example": "vpc-2zeuphj08tt7q3brd****"
},
"NatIpAddress": {
"description": "\u4e91\u4ea7\u54c1\u7684IP\uff0c\u7528\u4e8eVPC\u4e91\u4ea7\u54c1\u4e4b\u95f4\u7684\u7f51\u7edc\u4e92\u901a\u3002",
"type": "文本",
"example": "172.17.**.**"
},
"VSwitchId": {
"description": "\u865a\u62df\u4ea4\u6362\u673aID\u3002",
"type": "文本",
"example": "vsw-2zeh0r1pabwtg6wcs****"
},
"PrivateIpAddress": {
"type": "array",
"items": {
"description": "\u79c1\u6709IP\u5730\u5740\u3002",
"type": "文本",
"example": "172.17.**.**"
},
"description": "\u79c1\u6709IP\u5730\u5740\u5217\u8868\u3002"
}
},
"desc": "\u4e13\u6709\u7f51\u7edcVPC\u5c5e\u6027\u3002"
},
{
"name": "EipAddress",
"type": "json",
"example": {
"IsSupportUnassociate": {
"description": "\u662f\u5426\u53ef\u4ee5\u89e3\u7ed1\u5f39\u6027\u516c\u7f51IP\u3002",
"type": "boolean",
"example": "true"
},
"InternetChargeType": {
"description": "\u5f39\u6027\u516c\u7f51IP\u7684\u8ba1\u8d39\u65b9\u5f0f\u3002\n\n- PayByBandwidth\uff1a\u6309\u5e26\u5bbd\u8ba1\u8d39\u3002\n\n- PayByTraffic\uff1a\u6309\u6d41\u91cf\u8ba1\u8d39\u3002",
"type": "文本",
"example": "PayByTraffic"
},
"IpAddress": {
"description": "\u5f39\u6027\u516c\u7f51IP\u3002",
"type": "文本",
"example": "42.112.**.**"
},
"Bandwidth": {
"description": "\u5f39\u6027\u516c\u7f51IP\u7684\u516c\u7f51\u5e26\u5bbd\u9650\u901f\uff0c\u5355\u4f4d\u4e3aMbit/s\u3002",
"type": "整数",
"format": "int32",
"example": "5"
},
"AllocationId": {
"description": "\u5f39\u6027\u516c\u7f51IP\u7684ID\u3002",
"type": "文本",
"example": "eip-2ze88m67qx5z****"
}
},
"desc": "\u5f39\u6027\u516c\u7f51IP\u7ed1\u5b9a\u4fe1\u606f\u3002"
},
{
"name": "HibernationOptions",
"type": "json",
"example": {
"Configured": {
"description": "> \u8be5\u53c2\u6570\u6b63\u5728\u9080\u6d4b\u4e2d\uff0c\u6682\u672a\u5f00\u653e\u4f7f\u7528\u3002",
"type": "boolean",
"example": "false"
}
},
"desc": "> \u8be5\u53c2\u6570\u6b63\u5728\u9080\u6d4b\u4e2d\uff0c\u6682\u672a\u5f00\u653e\u4f7f\u7528\u3002"
},
{
"name": "DedicatedHostAttribute",
"type": "json",
"example": {
"DedicatedHostId": {
"description": "\u4e13\u6709\u5bbf\u4e3b\u673aID\u3002",
"type": "文本",
"example": "dh-bp67acfmxazb4p****"
},
"DedicatedHostName": {
"description": "\u4e13\u6709\u5bbf\u4e3b\u673a\u540d\u79f0\u3002",
"type": "文本",
"example": "testDedicatedHostName"
},
"DedicatedHostClusterId": {
"description": "\u4e13\u6709\u5bbf\u4e3b\u673a\u96c6\u7fa4ID\u3002",
"type": "文本",
"example": "dc-bp67acfmxazb4h****"
}
},
"desc": "\u7531\u4e13\u6709\u5bbf\u4e3b\u673a\u96c6\u7fa4ID\uff08DedicatedHostClusterId\uff09\u3001\u4e13\u6709\u5bbf\u4e3b\u673aID\uff08DedicatedHostId\uff09\u548c\u540d\u79f0\uff08DedicatedHostName\uff09\u7ec4\u6210\u7684\u5bbf\u4e3b\u673a\u5c5e\u6027\u6570\u7ec4\u3002"
},
{
"name": "EcsCapacityReservationAttr",
"type": "json",
"example": {
"CapacityReservationPreference": {
"description": "\u5bb9\u91cf\u9884\u7559\u504f\u597d\u3002",
"type": "文本",
"example": "cr-bp67acfmxazb4p****"
},
"CapacityReservationId": {
"description": "\u5bb9\u91cf\u9884\u7559ID\u3002",
"type": "文本",
"example": "cr-bp67acfmxazb4p****"
}
},
"desc": "\u4e91\u670d\u52a1\u5668ECS\u7684\u5bb9\u91cf\u9884\u7559\u76f8\u5173\u53c2\u6570\u3002"
},
{
"name": "DedicatedInstanceAttribute",
"type": "json",
"example": {
"Affinity": {
"description": "\u4e13\u6709\u5bbf\u4e3b\u673a\u5b9e\u4f8b\u662f\u5426\u4e0e\u4e13\u6709\u5bbf\u4e3b\u673a\u5173\u8054\u3002\u53ef\u80fd\u503c\uff1a\n\n- default\uff1a\u4e13\u6709\u5bbf\u4e3b\u673a\u5b9e\u4f8b\u4e0d\u4e0e\u4e13\u6709\u5bbf\u4e3b\u673a\u5173\u8054\u3002\u505c\u673a\u4e0d\u6536\u8d39\u5b9e\u4f8b\u91cd\u542f\u540e\uff0c\u53ef\u80fd\u4f1a\u653e\u7f6e\u5728\u81ea\u52a8\u8d44\u6e90\u90e8\u7f72\u6c60\u4e2d\u7684\u5176\u5b83\u4e13\u6709\u5bbf\u4e3b\u673a\u4e0a\u3002\n\n- host\uff1a\u4e13\u6709\u5bbf\u4e3b\u673a\u5b9e\u4f8b\u4e0e\u4e13\u6709\u5bbf\u4e3b\u673a\u5173\u8054\u3002\u505c\u673a\u4e0d\u6536\u8d39\u5b9e\u4f8b\u91cd\u542f\u540e\uff0c\u4ecd\u653e\u7f6e\u5728\u539f\u4e13\u6709\u5bbf\u4e3b\u673a\u4e0a\u3002",
"type": "文本",
"example": "default"
},
"Tenancy": {
"description": "\u5b9e\u4f8b\u7684\u5bbf\u4e3b\u673a\u7c7b\u578b\u662f\u5426\u4e3a\u4e13\u6709\u5bbf\u4e3b\u673a\u3002\u53ef\u80fd\u503c\uff1a\n\n- default\uff1a\u5b9e\u4f8b\u7684\u5bbf\u4e3b\u673a\u7c7b\u578b\u4e0d\u662f\u4e13\u6709\u5bbf\u4e3b\u673a\u3002\n\n- host\uff1a\u5b9e\u4f8b\u7684\u5bbf\u4e3b\u673a\u7c7b\u578b\u4e3a\u4e13\u6709\u5bbf\u4e3b\u673a\u3002",
"type": "文本",
"example": "default"
}
},
"desc": "\u4e13\u6709\u5bbf\u4e3b\u673a\u5b9e\u4f8b\u7684\u5c5e\u6027\u3002"
},
{
"name": "CpuOptions",
"type": "json",
"example": {
"Numa": {
"description": "\u5206\u914d\u7684\u7ebf\u7a0b\u6570\u3002\u53ef\u80fd\u503c\u4e3a2\u3002",
"type": "文本",
"example": "2"
},
"CoreCount": {
"description": "\u7269\u7406CPU\u6838\u5fc3\u6570\u3002",
"type": "整数",
"format": "int32",
"example": "2"
},
"ThreadsPerCore": {
"description": "CPU\u7ebf\u7a0b\u6570\u3002",
"type": "整数",
"format": "int32",
"example": "4"
}
},
"desc": "CPU\u914d\u7f6e\u8be6\u60c5\u3002"
},
{
"name": "MetadataOptions",
"type": "json",
"example": {
"HttpEndpoint": {
"description": "\u662f\u5426\u542f\u7528\u5b9e\u4f8b\u5143\u6570\u636e\u7684\u8bbf\u95ee\u901a\u9053\u3002\u53ef\u80fd\u503c\uff1a\n- enabled\uff1a\u542f\u7528\u3002\n- disabled\uff1a\u7981\u7528\u3002",
"type": "文本",
"example": "enabled"
},
"HttpPutResponseHopLimit": {
"description": "> \u8be5\u53c2\u6570\u6682\u672a\u5f00\u653e\u4f7f\u7528\u3002",
"type": "整数",
"format": "int32",
"example": "0"
},
"HttpTokens": {
"description": "\u8bbf\u95ee\u5b9e\u4f8b\u5143\u6570\u636e\u65f6\u662f\u5426\u5f3a\u5236\u4f7f\u7528\u52a0\u56fa\u6a21\u5f0f\uff08IMDSv2\uff09\u3002\u53ef\u80fd\u503c\uff1a\n- optional\uff1a\u4e0d\u5f3a\u5236\u4f7f\u7528\u3002\n- required\uff1a\u5f3a\u5236\u4f7f\u7528\u3002",
"type": "文本",
"example": "optional"
}
},
"desc": "\u5143\u6570\u636e\u9009\u9879\u96c6\u5408\u3002"
},
{
"name": "ImageOptions",
"type": "json",
"example": {
"LoginAsNonRoot": {
"description": "\u4f7f\u7528\u8be5\u955c\u50cf\u7684\u5b9e\u4f8b\u662f\u5426\u652f\u6301\u4f7f\u7528ecs-user\u7528\u6237\u767b\u5f55\u3002\u53ef\u80fd\u503c\uff1a\n\n- true\uff1a\u662f\n\n- false\uff1a\u5426",
"type": "boolean",
"example": "false"
}
},
"desc": "\u955c\u50cf\u76f8\u5173\u5c5e\u6027\u4fe1\u606f\u3002"
}
]

427
cmdb-api/api/lib/cmdb/auto_discovery/templates/aws_ec2.json Normal file
View File

@ -0,0 +1,427 @@
[
{
"name": "amiLaunchIndex",
"type": "整数",
"desc": "The AMI launch index, which can be used to find this instance in the launch group.",
"example": "0"
},
{
"name": "architecture",
"type": "文本",
"desc": "The architecture of the image.",
"example": "x86_64"
},
{
"name": "blockDeviceMapping",
"type": "json",
"desc": "Any block device mapping entries for the instance.",
"example": {
"item": {
"deviceName": "/dev/xvda",
"ebs": {
"volumeId": "vol-1234567890abcdef0",
"status": "attached",
"attachTime": "2015-12-22T10:44:09.000Z",
"deleteOnTermination": "true"
}
}
}
},
{
"name": "bootMode",
"type": "文本",
"desc": "The boot mode that was specified by the AMI. If the value is uefi-preferred, the AMI supports both UEFI and Legacy BIOS. The currentInstanceBootMode parameter is the boot mode that is used to boot the instance at launch or start.",
"example": null
},
{
"name": "capacityReservationId",
"type": "文本",
"desc": "The ID of the Capacity Reservation.",
"example": null
},
{
"name": "capacityReservationSpecification",
"type": "json",
"desc": "Information about the Capacity Reservation targeting option.",
"example": null
},
{
"name": "clientToken",
"type": "文本",
"desc": "The idempotency token you provided when you launched the instance, if applicable.",
"example": "xMcwG14507example"
},
{
"name": "cpuOptions",
"type": "json",
"desc": "The CPU options for the instance.",
"example": {
"coreCount": "1",
"threadsPerCore": "1"
}
},
{
"name": "currentInstanceBootMode",
"type": "文本",
"desc": "The boot mode that is used to boot the instance at launch or start. For more information, see Boot modes in the Amazon EC2 User Guide.",
"example": null
},
{
"name": "dnsName",
"type": "文本",
"desc": "[IPv4 only] The public DNS name assigned to the instance. This name is not available until the instance enters the running state. This name is only available if you've enabled DNS hostnames for your VPC.",
"example": "ec2-54-194-252-215.eu-west-1.compute.amazonaws.com"
},
{
"name": "ebsOptimized",
"type": "Boolean",
"desc": "Indicates whether the instance is optimized for Amazon EBS I/O. This optimization provides dedicated throughput to Amazon EBS and an optimized configuration stack to provide optimal I/O performance. This optimization isn't available with all instance types. Additional usage charges apply when using an EBS Optimized instance.",
"example": "false"
},
{
"name": "elasticGpuAssociationSet",
"type": "json",
"desc": "The Elastic GPU associated with the instance.",
"example": null
},
{
"name": "elasticInferenceAcceleratorAssociationSet",
"type": "json",
"desc": "The elastic inference accelerator associated with the instance.",
"example": null
},
{
"name": "enaSupport",
"type": "Boolean",
"desc": "Specifies whether enhanced networking with ENA is enabled.",
"example": null
},
{
"name": "enclaveOptions",
"type": "json",
"desc": "Indicates whether the instance is enabled for AWS Nitro Enclaves.",
"example": null
},
{
"name": "groupSet",
"type": "json",
"desc": "The security groups for the instance.",
"example": {
"item": {
"groupId": "sg-e4076980",
"groupName": "SecurityGroup1"
}
}
},
{
"name": "hibernationOptions",
"type": "json",
"desc": "Indicates whether the instance is enabled for hibernation.",
"example": null
},
{
"name": "hypervisor",
"type": "文本",
"desc": "The hypervisor type of the instance. The value xen is used for both Xen and Nitro hypervisors.",
"example": "xen"
},
{
"name": "iamInstanceProfile",
"type": "json",
"desc": "The IAM instance profile associated with the instance, if applicable.",
"example": {
"arn": "arn:aws:iam::123456789012:instance-profile/AdminRole",
"id": "ABCAJEDNCAA64SSD123AB"
}
},
{
"name": "imageId",
"type": "文本",
"desc": "The ID of the AMI used to launch the instance.",
"example": "ami-bff32ccc"
},
{
"name": "instanceId",
"type": "文本",
"desc": "The ID of the instance.",
"example": "i-1234567890abcdef0"
},
{
"name": "instanceLifecycle",
"type": "文本",
"desc": "Indicates whether this is a Spot Instance or a Scheduled Instance.",
"example": null
},
{
"name": "instanceState",
"type": "json",
"desc": "The current state of the instance.",
"example": {
"code": "16",
"name": "running"
}
},
{
"name": "instanceType",
"type": "文本",
"desc": "The instance type.",
"example": "t2.micro"
},
{
"name": "ipAddress",
"type": "文本",
"desc": "The public IPv4 address, or the Carrier IP address assigned to the instance, if applicable.",
"example": "54.194.252.215"
},
{
"name": "ipv6Address",
"type": "文本",
"desc": "The IPv6 address assigned to the instance.",
"example": null
},
{
"name": "kernelId",
"type": "文本",
"desc": "The kernel associated with this instance, if applicable.",
"example": null
},
{
"name": "keyName",
"type": "文本",
"desc": "The name of the key pair, if this instance was launched with an associated key pair.",
"example": "my_keypair"
},
{
"name": "launchTime",
"type": "Time",
"desc": "The time the instance was launched.",
"example": "2018-05-08T16:46:19.000Z"
},
{
"name": "licenseSet",
"type": "json",
"desc": "The license configurations for the instance.",
"example": null
},
{
"name": "maintenanceOptions",
"type": "json",
"desc": "Provides information on the recovery and maintenance options of your instance.",
"example": null
},
{
"name": "metadataOptions",
"type": "json",
"desc": "The metadata options for the instance.",
"example": null
},
{
"name": "monitoring",
"type": "json",
"desc": "The monitoring for the instance.",
"example": {
"state": "disabled"
}
},
{
"name": "networkInterfaceSet",
"type": "json",
"desc": "The network interfaces for the instance.",
"example": {
"item": {
"networkInterfaceId": "eni-551ba033",
"subnetId": "subnet-56f5f633",
"vpcId": "vpc-11112222",
"description": "Primary network interface",
"ownerId": "123456789012",
"status": "in-use",
"macAddress": "02:dd:2c:5e:01:69",
"privateIpAddress": "192.168.1.88",
"privateDnsName": "ip-192-168-1-88.eu-west-1.compute.internal",
"sourceDestCheck": "true",
"groupSet": {
"item": {
"groupId": "sg-e4076980",
"groupName": "SecurityGroup1"
}
},
"attachment": {
"attachmentId": "eni-attach-39697adc",
"deviceIndex": "0",
"status": "attached",
"attachTime": "2018-05-08T16:46:19.000Z",
"deleteOnTermination": "true"
},
"association": {
"publicIp": "54.194.252.215",
"publicDnsName": "ec2-54-194-252-215.eu-west-1.compute.amazonaws.com",
"ipOwnerId": "amazon"
},
"privateIpAddressesSet": {
"item": {
"privateIpAddress": "192.168.1.88",
"privateDnsName": "ip-192-168-1-88.eu-west-1.compute.internal",
"primary": "true",
"association": {
"publicIp": "54.194.252.215",
"publicDnsName": "ec2-54-194-252-215.eu-west-1.compute.amazonaws.com",
"ipOwnerId": "amazon"
}
}
},
"ipv6AddressesSet": {
"item": {
"ipv6Address": "2001:db8:1234:1a2b::123"
}
}
}
}
},
{
"name": "outpostArn",
"type": "文本",
"desc": "The Amazon Resource Name (ARN) of the Outpost.",
"example": null
},
{
"name": "placement",
"type": "json",
"desc": "The location where the instance launched, if applicable.",
"example": {
"availabilityZone": "eu-west-1c",
"groupName": null,
"tenancy": "default"
}
},
{
"name": "platform",
"type": "文本",
"desc": "The value is Windows for Windows instances; otherwise blank.",
"example": null
},
{
"name": "platformDetails",
"type": "文本",
"desc": "The platform details value for the instance. For more information, see AMI billing information fields in the Amazon EC2 User Guide.",
"example": null
},
{
"name": "privateDnsName",
"type": "文本",
"desc": "[IPv4 only] The private DNS hostname name assigned to the instance. This DNS hostname can only be used inside the Amazon EC2 network. This name is not available until the instance enters the running state.",
"example": "ip-192-168-1-88.eu-west-1.compute.internal"
},
{
"name": "privateDnsNameOptions",
"type": "json",
"desc": "The options for the instance hostname.",
"example": null
},
{
"name": "privateIpAddress",
"type": "文本",
"desc": "The private IPv4 address assigned to the instance.",
"example": "192.168.1.88"
},
{
"name": "productCodes",
"type": "json",
"desc": "The product codes attached to this instance, if applicable.",
"example": null
},
{
"name": "ramdiskId",
"type": "文本",
"desc": "The RAM disk associated with this instance, if applicable.",
"example": null
},
{
"name": "reason",
"type": "文本",
"desc": "The reason for the most recent state transition. This might be an empty string.",
"example": null
},
{
"name": "rootDeviceName",
"type": "文本",
"desc": "The device name of the root device volume (for example, /dev/sda1).",
"example": "/dev/xvda"
},
{
"name": "rootDeviceType",
"type": "文本",
"desc": "The root device type used by the AMI. The AMI can use an EBS volume or an instance store volume.",
"example": "ebs"
},
{
"name": "sourceDestCheck",
"type": "Boolean",
"desc": "Indicates whether source/destination checking is enabled.",
"example": "true"
},
{
"name": "spotInstanceRequestId",
"type": "文本",
"desc": "If the request is a Spot Instance request, the ID of the request.",
"example": null
},
{
"name": "sriovNetSupport",
"type": "文本",
"desc": "Specifies whether enhanced networking with the Intel 82599 Virtual Function interface is enabled.",
"example": null
},
{
"name": "stateReason",
"type": "json",
"desc": "The reason for the most recent state transition.",
"example": null
},
{
"name": "subnetId",
"type": "文本",
"desc": "The ID of the subnet in which the instance is running.",
"example": "subnet-56f5f633"
},
{
"name": "tagSet",
"type": "json",
"desc": "Any tags assigned to the instance.",
"example": {
"item": {
"key": "Name",
"value": "Server_1"
}
}
},
{
"name": "tpmSupport",
"type": "文本",
"desc": "If the instance is configured for NitroTPM support, the value is v2.0. For more information, see NitroTPM in the Amazon EC2 User Guide.",
"example": null
},
{
"name": "usageOperation",
"type": "文本",
"desc": "The usage operation value for the instance. For more information, see AMI billing information fields in the Amazon EC2 User Guide.",
"example": null
},
{
"name": "usageOperationUpdateTime",
"type": "Time",
"desc": "The time that the usage operation was last updated.",
"example": null
},
{
"name": "virtualizationType",
"type": "文本",
"desc": "The virtualization type of the instance.",
"example": "hvm"
},
{
"name": "vpcId",
"type": "文本",
"desc": "The ID of the VPC in which the instance is running.",
"example": "vpc-11112222"
}
]

292
cmdb-api/api/lib/cmdb/auto_discovery/templates/huaweicloud_ecs.json Normal file
View File

@ -0,0 +1,292 @@
[
{
"name": "status",
"type": "文本",
"example": "ACTIVE",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u72b6\u6001\u3002\n\n\u53d6\u503c\u8303\u56f4:\n\nACTIVE\u3001BUILD\u3001DELETED\u3001ERROR\u3001HARD_REBOOT\u3001MIGRATING\u3001PAUSED\u3001REBOOT\u3001REBUILD\u3001RESIZE\u3001REVERT_RESIZE\u3001SHUTOFF\u3001SHELVED\u3001SHELVED_OFFLOADED\u3001SOFT_DELETED\u3001SUSPENDED\u3001VERIFY_RESIZE\n\n\u5f39\u6027\u4e91\u670d\u52a1\u5668\u72b6\u6001\u8bf4\u660e\u8bf7\u53c2\u8003[\u4e91\u670d\u52a1\u5668\u72b6\u6001](https://support.huaweicloud.com/api-ecs/ecs_08_0002.html)"
},
{
"name": "updated",
"type": "文本",
"example": "2019-05-22T03:30:52Z",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u66f4\u65b0\u65f6\u95f4\u3002\n\n\u65f6\u95f4\u683c\u5f0f\u4f8b\u5982:2019-05-22T03:30:52Z"
},
{
"name": "auto_terminate_time",
"type": "文本",
"example": "2020-01-19T03:30:52Z",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u81ea\u52a8\u91ca\u653e\u65f6\u95f4\u3002\n\n\u65f6\u95f4\u683c\u5f0f\u4f8b\u5982:2020-01-19T03:30:52Z"
},
{
"name": "hostId",
"type": "文本",
"example": "c7145889b2e3202cd295ceddb1742ff8941b827b586861fd0acedf64",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u6240\u5728\u4e3b\u673a\u7684\u4e3b\u673aID\u3002"
},
{
"name": "OS-EXT-SRV-ATTR:host",
"type": "文本",
"example": "pod01.cn-north-1c",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u6240\u5728\u4e3b\u673a\u7684\u4e3b\u673a\u540d\u79f0\u3002"
},
{
"name": "addresses",
"type": "json",
"example": null,
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7684\u7f51\u7edc\u5c5e\u6027\u3002"
},
{
"name": "key_name",
"type": "文本",
"example": "KeyPair-test",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u4f7f\u7528\u7684\u5bc6\u94a5\u5bf9\u540d\u79f0\u3002"
},
{
"name": "image",
"type": "json",
"example": null,
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u955c\u50cf\u4fe1\u606f\u3002"
},
{
"name": "OS-EXT-STS:task_state",
"type": "文本",
"example": "rebooting",
"desc": "\u6269\u5c55\u5c5e\u6027,\u5f39\u6027\u4e91\u670d\u52a1\u5668\u5f53\u524d\u4efb\u52a1\u7684\u72b6\u6001\u3002\n\n\u53d6\u503c\u8303\u56f4\u8bf7\u53c2\u8003[\u4e91\u670d\u52a1\u5668\u72b6\u6001](https://support.huaweicloud.com/api-ecs/ecs_08_0002.html)\u88683\u3002"
},
{
"name": "OS-EXT-STS:vm_state",
"type": "文本",
"example": "active",
"desc": "\u6269\u5c55\u5c5e\u6027,\u5f39\u6027\u4e91\u670d\u52a1\u5668\u5f53\u524d\u72b6\u6001\u3002\n\n\u4e91\u670d\u52a1\u5668\u72b6\u6001\u8bf4\u660e\u8bf7\u53c2\u8003[\u4e91\u670d\u52a1\u5668\u72b6\u6001](https://support.huaweicloud.com/api-ecs/ecs_08_0002.html)\u3002"
},
{
"name": "OS-EXT-SRV-ATTR:instance_name",
"type": "文本",
"example": "instance-0048a91b",
"desc": "\u6269\u5c55\u5c5e\u6027,\u5f39\u6027\u4e91\u670d\u52a1\u5668\u522b\u540d\u3002"
},
{
"name": "OS-EXT-SRV-ATTR:hypervisor_hostname",
"type": "文本",
"example": "nova022@36",
"desc": "\u6269\u5c55\u5c5e\u6027,\u5f39\u6027\u4e91\u670d\u52a1\u5668\u6240\u5728\u865a\u62df\u5316\u4e3b\u673a\u540d\u3002"
},
{
"name": "flavor",
"type": "json",
"example": null,
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u89c4\u683c\u4fe1\u606f\u3002"
},
{
"name": "id",
"type": "文本",
"example": "4f4b3dfa-eb70-47cf-a60a-998a53bd6666",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668ID,\u683c\u5f0f\u4e3aUUID\u3002"
},
{
"name": "security_groups",
"type": "json",
"example": {
"$ref": "#/definitions/ServerSecurityGroup"
},
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u6240\u5c5e\u5b89\u5168\u7ec4\u5217\u8868\u3002"
},
{
"name": "OS-EXT-AZ:availability_zone",
"type": "文本",
"example": "cn-north-1c",
"desc": "\u6269\u5c55\u5c5e\u6027,\u5f39\u6027\u4e91\u670d\u52a1\u5668\u6240\u5728\u53ef\u7528\u533a\u540d\u79f0\u3002"
},
{
"name": "user_id",
"type": "文本",
"example": "05498fe56b8010d41f7fc01e280b6666",
"desc": "\u521b\u5efa\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7684\u7528\u6237ID,\u683c\u5f0f\u4e3aUUID\u3002"
},
{
"name": "name",
"type": "文本",
"example": "ecs-test-server",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u540d\u79f0\u3002"
},
{
"name": "created",
"type": "文本",
"example": "2017-07-15T11:30:52Z",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u521b\u5efa\u65f6\u95f4\u3002\n\n\u65f6\u95f4\u683c\u5f0f\u4f8b\u5982:2019-05-22T03:19:19Z"
},
{
"name": "tenant_id",
"type": "文本",
"example": "743b4c0428d94531b9f2add666646666",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u6240\u5c5e\u79df\u6237ID,\u5373\u9879\u76eeid,\u548cproject_id\u8868\u793a\u76f8\u540c\u7684\u6982\u5ff5,\u683c\u5f0f\u4e3aUUID\u3002"
},
{
"name": "OS-DCF:diskConfig",
"type": "文本",
"example": "AUTO",
"desc": "\u6269\u5c55\u5c5e\u6027, diskConfig\u7684\u7c7b\u578b\u3002\n\n- MANUAL,\u955c\u50cf\u7a7a\u95f4\u4e0d\u4f1a\u6269\u5c55\u3002\n- AUTO,\u7cfb\u7edf\u76d8\u955c\u50cf\u7a7a\u95f4\u4f1a\u81ea\u52a8\u6269\u5c55\u4e3a\u4e0eflavor\u5927\u5c0f\u4e00\u81f4\u3002"
},
{
"name": "accessIPv4",
"type": "文本",
"example": null,
"desc": "\u9884\u7559\u5c5e\u6027\u3002"
},
{
"name": "accessIPv6",
"type": "文本",
"example": null,
"desc": "\u9884\u7559\u5c5e\u6027\u3002"
},
{
"name": "fault",
"type": "文本",
"example": null,
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u6545\u969c\u4fe1\u606f\u3002\n\n\u53ef\u9009\u53c2\u6570,\u5728\u5f39\u6027\u4e91\u670d\u52a1\u5668\u72b6\u6001\u4e3aERROR\u4e14\u5b58\u5728\u5f02\u5e38\u7684\u60c5\u51b5\u4e0b\u8fd4\u56de\u3002"
},
{
"name": "progress",
"type": "整数",
"example": null,
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u8fdb\u5ea6\u3002"
},
{
"name": "OS-EXT-STS:power_state",
"type": "整数",
"example": 4,
"desc": "\u6269\u5c55\u5c5e\u6027,\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7535\u6e90\u72b6\u6001\u3002"
},
{
"name": "config_drive",
"type": "文本",
"example": null,
"desc": "config drive\u4fe1\u606f\u3002"
},
{
"name": "metadata",
"type": "json",
"example": null,
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u5143\u6570\u636e\u3002\n\n> \u8bf4\u660e:\n> \n> \u5143\u6570\u636e\u5305\u542b\u7cfb\u7edf\u9ed8\u8ba4\u6dfb\u52a0\u5b57\u6bb5\u548c\u7528\u6237\u8bbe\u7f6e\u7684\u5b57\u6bb5\u3002\n\n\u7cfb\u7edf\u9ed8\u8ba4\u6dfb\u52a0\u5b57\u6bb5\n\n1. charging_mode\n\u4e91\u670d\u52a1\u5668\u7684\u8ba1\u8d39\u7c7b\u578b\u3002\n\n- \u201c0\u201d:\u6309\u9700\u8ba1\u8d39(\u5373postPaid-\u540e\u4ed8\u8d39\u65b9\u5f0f)\u3002\n- \u201c1\u201d:\u6309\u5305\u5e74\u5305\u6708\u8ba1\u8d39(\u5373prePaid-\u9884\u4ed8\u8d39\u65b9\u5f0f)\u3002\"2\":\u7ade\u4ef7\u5b9e\u4f8b\u8ba1\u8d39\n\n2. metering.order_id\n\u6309\u201c\u5305\u5e74/\u5305\u6708\u201d\u8ba1\u8d39\u7684\u4e91\u670d\u52a1\u5668\u5bf9\u5e94\u7684\u8ba2\u5355ID\u3002\n\n3. metering.product_id\n\u6309\u201c\u5305\u5e74/\u5305\u6708\u201d\u8ba1\u8d39\u7684\u4e91\u670d\u52a1\u5668\u5bf9\u5e94\u7684\u4ea7\u54c1ID\u3002\n\n4. vpc_id\n\u4e91\u670d\u52a1\u5668\u6240\u5c5e\u7684\u865a\u62df\u79c1\u6709\u4e91ID\u3002\n\n5. EcmResStatus\n\u4e91\u670d\u52a1\u5668\u7684\u51bb\u7ed3\u72b6\u6001\u3002\n\n- normal:\u4e91\u670d\u52a1\u5668\u6b63\u5e38\u72b6\u6001(\u672a\u88ab\u51bb\u7ed3)\u3002\n- freeze:\u4e91\u670d\u52a1\u5668\u88ab\u51bb\u7ed3\u3002\n\n> \u5f53\u4e91\u670d\u52a1\u5668\u88ab\u51bb\u7ed3\u6216\u8005\u89e3\u51bb\u540e,\u7cfb\u7edf\u9ed8\u8ba4\u6dfb\u52a0\u8be5\u5b57\u6bb5,\u4e14\u8be5\u5b57\u6bb5\u5fc5\u9009\u3002\n\n6. metering.image_id\n\u4e91\u670d\u52a1\u5668\u64cd\u4f5c\u7cfb\u7edf\u5bf9\u5e94\u7684\u955c\u50cfID\n\n7. metering.imagetype\n\u955c\u50cf\u7c7b\u578b,\u76ee\u524d\u652f\u6301:\n\n- \u516c\u5171\u955c\u50cf(gold)\n- \u79c1\u6709\u955c\u50cf(private)\n- \u5171\u4eab\u955c\u50cf(shared)\n\n8. metering.resourcespeccode\n\u4e91\u670d\u52a1\u5668\u5bf9\u5e94\u7684\u8d44\u6e90\u89c4\u683c\u3002\n\n9. image_name\n\u4e91\u670d\u52a1\u5668\u64cd\u4f5c\u7cfb\u7edf\u5bf9\u5e94\u7684\u955c\u50cf\u540d\u79f0\u3002\n\n10. os_bit\n\u64cd\u4f5c\u7cfb\u7edf\u4f4d\u6570,\u4e00\u822c\u53d6\u503c\u4e3a\u201c32\u201d\u6216\u8005\u201c64\u201d\u3002\n\n11. lockCheckEndpoint\n\u56de\u8c03URL,\u7528\u4e8e\u68c0\u67e5\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7684\u52a0\u9501\u662f\u5426\u6709\u6548\u3002\n\n- \u5982\u679c\u6709\u6548,\u5219\u4e91\u670d\u52a1\u5668\u4fdd\u6301\u9501\u5b9a\u72b6\u6001\u3002\n- \u5982\u679c\u65e0\u6548,\u89e3\u9664\u9501\u5b9a\u72b6\u6001,\u5220\u9664\u5931\u6548\u7684\u9501\u3002\n\n12. lockSource\n\u5f39\u6027\u4e91\u670d\u52a1\u5668\u6765\u81ea\u54ea\u4e2a\u670d\u52a1\u3002\u8ba2\u5355\u52a0\u9501(ORDER)\n\n13. lockSourceId\n\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7684\u52a0\u9501\u6765\u81ea\u54ea\u4e2aID\u3002lockSource\u4e3a\u201cORDER\u201d\u65f6,lockSourceId\u4e3a\u8ba2\u5355ID\u3002\n\n14. lockScene\n\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7684\u52a0\u9501\u7c7b\u578b\u3002\n\n- \u6309\u9700\u8f6c\u5305\u5468\u671f(TO_PERIOD_LOCK)\n\n15. virtual_env_type\n\n- IOS\u955c\u50cf\u521b\u5efa\u865a\u62df\u673a,\"virtual_env_type\": \"IsoImage\" \u5c5e\u6027;\n- \u975eIOS\u955c\u50cf\u521b\u5efa\u865a\u62df\u673a,\u572819.5.0\u7248\u672c\u4ee5\u540e\u521b\u5efa\u7684\u865a\u62df\u673a\u5c06\u4e0d\u4f1a\u6dfb\u52a0virtual_env_type \u5c5e\u6027,\u800c\u5728\u6b64\u4e4b\u524d\u7684\u7248\u672c\u521b\u5efa\u7684\u865a\u62df\u673a\u53ef\u80fd\u4f1a\u8fd4\u56de\"virtual_env_type\": \"FusionCompute\"\u5c5e\u6027 \u3002\n\n> virtual_env_type\u5c5e\u6027\u4e0d\u5141\u8bb8\u7528\u6237\u589e\u52a0\u3001\u5220\u9664\u548c\u4fee\u6539\u3002\n\n16. metering.resourcetype\n\u4e91\u670d\u52a1\u5668\u5bf9\u5e94\u7684\u8d44\u6e90\u7c7b\u578b\u3002\n\n17. os_type\n\u64cd\u4f5c\u7cfb\u7edf\u7c7b\u578b,\u53d6\u503c\u4e3a:Linux\u3001Windows\u3002\n\n18. cascaded.instance_extrainfo\n\u7cfb\u7edf\u5185\u90e8\u865a\u62df\u673a\u6269\u5c55\u4fe1\u606f\u3002\n\n19. __support_agent_list\n\u4e91\u670d\u52a1\u5668\u662f\u5426\u652f\u6301\u4f01\u4e1a\u4e3b\u673a\u5b89\u5168\u3001\u4e3b\u673a\u76d1\u63a7\u3002\n\n- \u201chss\u201d:\u4f01\u4e1a\u4e3b\u673a\u5b89\u5168\n- \u201cces\u201d:\u4e3b\u673a\u76d1\u63a7\n\n20. agency_name\n\u59d4\u6258\u7684\u540d\u79f0\u3002\n\n\u59d4\u6258\u662f\u7531\u79df\u6237\u7ba1\u7406\u5458\u5728\u7edf\u4e00\u8eab\u4efd\u8ba4\u8bc1\u670d\u52a1(Identity and Access Management,IAM)\u4e0a\u521b\u5efa\u7684,\u53ef\u4ee5\u4e3a\u5f39\u6027\u4e91\u670d\u52a1\u5668\u63d0\u4f9b\u8bbf\u95ee\u4e91\u670d\u52a1\u7684\u4e34\u65f6\u51ed\u8bc1\u3002"
},
{
"name": "OS-SRV-USG:launched_at",
"type": "文本",
"example": "2018-08-15T14:21:22.000000",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u542f\u52a8\u65f6\u95f4\u3002\u65f6\u95f4\u683c\u5f0f\u4f8b\u5982:2019-05-22T03:23:59.000000"
},
{
"name": "OS-SRV-USG:terminated_at",
"type": "文本",
"example": "2019-05-22T03:23:59.000000",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u5220\u9664\u65f6\u95f4\u3002\n\n\u65f6\u95f4\u683c\u5f0f\u4f8b\u5982:2019-05-22T03:23:59.000000"
},
{
"name": "os-extended-volumes:volumes_attached",
"type": "json",
"example": {
"$ref": "#/definitions/ServerExtendVolumeAttachment"
},
"desc": "\u6302\u8f7d\u5230\u5f39\u6027\u4e91\u670d\u52a1\u5668\u4e0a\u7684\u78c1\u76d8\u3002"
},
{
"name": "description",
"type": "文本",
"example": "ecs description",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7684\u63cf\u8ff0\u4fe1\u606f\u3002"
},
{
"name": "host_status",
"type": "文本",
"example": "UP",
"desc": "nova-compute\u72b6\u6001\u3002\n\n- UP:\u670d\u52a1\u6b63\u5e38\n- UNKNOWN:\u72b6\u6001\u672a\u77e5\n- DOWN:\u670d\u52a1\u5f02\u5e38\n- MAINTENANCE:\u7ef4\u62a4\u72b6\u6001\n- \u7a7a\u5b57\u7b26\u4e32:\u5f39\u6027\u4e91\u670d\u52a1\u5668\u65e0\u4e3b\u673a\u4fe1\u606f"
},
{
"name": "OS-EXT-SRV-ATTR:hostname",
"type": "文本",
"example": null,
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7684\u4e3b\u673a\u540d\u3002"
},
{
"name": "OS-EXT-SRV-ATTR:reservation_id",
"type": "文本",
"example": "r-f06p3js8",
"desc": "\u6279\u91cf\u521b\u5efa\u573a\u666f,\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7684\u9884\u7559ID\u3002"
},
{
"name": "OS-EXT-SRV-ATTR:launch_index",
"type": "整数",
"example": null,
"desc": "\u6279\u91cf\u521b\u5efa\u573a\u666f,\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7684\u542f\u52a8\u987a\u5e8f\u3002"
},
{
"name": "OS-EXT-SRV-ATTR:kernel_id",
"type": "文本",
"example": null,
"desc": "\u82e5\u4f7f\u7528AMI\u683c\u5f0f\u7684\u955c\u50cf,\u5219\u8868\u793akernel image\u7684UUID;\u5426\u5219,\u7559\u7a7a\u3002"
},
{
"name": "OS-EXT-SRV-ATTR:ramdisk_id",
"type": "文本",
"example": null,
"desc": "\u82e5\u4f7f\u7528AMI\u683c\u5f0f\u955c\u50cf,\u5219\u8868\u793aramdisk image\u7684UUID;\u5426\u5219,\u7559\u7a7a\u3002"
},
{
"name": "OS-EXT-SRV-ATTR:root_device_name",
"type": "文本",
"example": "/dev/vda",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7cfb\u7edf\u76d8\u7684\u8bbe\u5907\u540d\u79f0\u3002"
},
{
"name": "OS-EXT-SRV-ATTR:user_data",
"type": "文本",
"example": "IyEvYmluL2Jhc2gKZWNobyAncm9vdDokNiRjcGRkSjckWm5WZHNiR253Z0l0SGlxUjZxbWtLTlJaeU9lZUtKd3dPbG9XSFdUeGFzWjA1STYwdnJYRTdTUTZGbEpFbWlXZ21WNGNmZ1pac1laN1BkMTBLRndyeC8nIHwgY2hwYXNzd2Q6666",
"desc": "\u521b\u5efa\u5f39\u6027\u4e91\u670d\u52a1\u5668\u65f6\u6307\u5b9a\u7684user_data\u3002"
},
{
"name": "locked",
"type": "boolean",
"example": null,
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u662f\u5426\u4e3a\u9501\u5b9a\u72b6\u6001\u3002\n\n- true:\u9501\u5b9a\n- false:\u672a\u9501\u5b9a"
},
{
"name": "tags",
"type": "文本、多值",
"example": {
"type": "文本"
},
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u6807\u7b7e\u3002"
},
{
"name": "os:scheduler_hints",
"type": "json",
"example": null,
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u8c03\u5ea6\u4fe1\u606f"
},
{
"name": "enterprise_project_id",
"type": "文本",
"example": "0",
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u6240\u5c5e\u7684\u4f01\u4e1a\u9879\u76eeID\u3002"
},
{
"name": "sys_tags",
"type": "文本、多值",
"example": {
"$ref": "#/definitions/ServerSystemTag"
},
"desc": "\u5f39\u6027\u4e91\u670d\u52a1\u5668\u7cfb\u7edf\u6807\u7b7e\u3002"
},
{
"name": "cpu_options",
"type": "json",
"example": null,
"desc": "\u81ea\u5b9a\u4e49CPU\u9009\u9879\u3002"
},
{
"name": "hypervisor",
"type": "文本",
"example": null,
"desc": "hypervisor\u4fe1\u606f\u3002"
}
]

37
cmdb-api/api/lib/cmdb/auto_discovery/templates/net_device.json Normal file
View File

@ -0,0 +1,37 @@
[{
"name":"manufacturer",
"type": "文本",
"example":"HUAWEI Technology Co.,Ltd",
"desc":"制造产商"
},{
"name":"sn",
"type": "文本",
"example":"102030059898",
"desc":"设备序列号"
},{
"name":"device_name",
"type": "文本",
"example":"USG6525E",
"desc":"设备名称"
},{
"name":"device_model",
"type": "文本",
"example":"2011.2.321.1.205",
"desc":"设备细分类型 结合相关产商获取相应的产品类型"
},{
"name":"description",
"type": "文本",
"example":"Huawei Vwersatile Routing Platform Software",
"desc":"设备描述"
},{
"name":"manager_ip",
"type": "文本",
"example":"192.168.1.1",
"desc":"管理ip"
}, {
"name":"ips",
"type": "文本、多值",
"example":"192.168.1.1, 192.168.1.2",
"desc":"ips"
}
]

297
cmdb-api/api/lib/cmdb/auto_discovery/templates/tencent_cvm.json Normal file
View File

@ -0,0 +1,297 @@
[
{
"name": "Placement",
"type": "json",
"desc": "实例所在的位置。",
"example": {
"HostId": "host-h3m57oik",
"ProjectId": 1174660,
"HostIds": [],
"Zone": "ap-guangzhou-1",
"HostIps": []
}
},
{
"name": "InstanceId",
"type": "文本",
"desc": "实例ID。",
"example": "ins-xlsyru2j"
},
{
"name": "InstanceType",
"type": "文本",
"desc": "实例机型。",
"example": "S2.SMALL2"
},
{
"name": "CPU",
"type": "整数",
"desc": "实例的CPU核数，单位：核。",
"example": 1
},
{
"name": "Memory",
"type": "整数",
"desc": "实例内存容量，单位：GB。",
"example": 1
},
{
"name": "RestrictState",
"type": "文本",
"desc": "实例业务状态。取值范围： NORMAL：表示正常状态的实例 EXPIRED：表示过期的实例 PROTECTIVELY_ISOLATED：表示被安全隔离的实例。",
"example": "PROTECTIVELY_ISOLATED"
},
{
"name": "InstanceName",
"type": "文本",
"desc": "实例名称。",
"example": "test"
},
{
"name": "InstanceChargeType",
"type": "文本",
"desc": "实例计费模式。取值范围： PREPAID：表示预付费，即包年包月 POSTPAID_BY_HOUR：表示后付费，即按量计费 CDHPAID：专用宿主机付费，即只对专用宿主机计费，不对专用宿主机上的实例计费。 SPOTPAID：表示竞价实例付费。",
"example": "POSTPAID_BY_HOUR"
},
{
"name": "SystemDisk",
"type": "json",
"desc": "实例系统盘信息。",
"example": {
"DiskSize": 50,
"CdcId": null,
"DiskId": "disk-czsodtl1",
"DiskType": "CLOUD_SSD"
}
},
{
"name": "DataDisks",
"type": "json",
"desc": "实例数据盘信息。",
"example": [
{
"DeleteWithInstance": true,
"Encrypt": true,
"CdcId": null,
"DiskType": "CLOUD_SSD",
"ThroughputPerformance": 0,
"KmsKeyId": null,
"DiskSize": 50,
"SnapshotId": null,
"DiskId": "disk-bzsodtn1"
}
]
},
{
"name": "PrivateIpAddresses",
"type": "文本、多值",
"desc": "实例主网卡的内网IP列表。",
"example": [
"172.16.32.78"
]
},
{
"name": "PublicIpAddresses",
"type": "文本、多值",
"desc": "实例主网卡的公网IP列表。 注意：此字段可能返回 null，表示取不到有效值。",
"example": [
"123.207.11.190"
]
},
{
"name": "InternetAccessible",
"type": "json",
"desc": "实例带宽信息。",
"example": {
"PublicIpAssigned": true,
"InternetChargeType": "TRAFFIC_POSTPAID_BY_HOUR",
"BandwidthPackageId": null,
"InternetMaxBandwidthOut": 1
}
},
{
"name": "VirtualPrivateCloud",
"type": "json",
"desc": "实例所属虚拟私有网络信息。",
"example": {
"SubnetId": "subnet-mv4sn55k",
"AsVpcGateway": false,
"Ipv6AddressCount": 1,
"VpcId": "vpc-m0cnatxj",
"PrivateIpAddresses": [
"172.16.3.59"
]
}
},
{
"name": "ImageId",
"type": "文本",
"desc": "生产实例所使用的镜像ID。",
"example": "img-8toqc6s3"
},
{
"name": "RenewFlag",
"type": "文本",
"desc": "自动续费标识。取值范围： NOTIFY_AND_MANUAL_RENEW：表示通知即将过期，但不自动续费 NOTIFY_AND_AUTO_RENEW：表示通知即将过期，而且自动续费 DISABLE_NOTIFY_AND_MANUAL_RENEW：表示不通知即将过期，也不自动续费。 注意：后付费模式本项为null",
"example": "NOTIFY_AND_MANUAL_RENEW"
},
{
"name": "CreatedTime",
"type": "json",
"desc": "创建时间。按照ISO8601标准表示，并且使用UTC时间。格式为：YYYY-MM-DDThh:mm:ssZ。",
"example": "2020-09-22T00:00:00+00:00"
},
{
"name": "ExpiredTime",
"type": "json",
"desc": "到期时间。按照ISO8601标准表示，并且使用UTC时间。格式为：YYYY-MM-DDThh:mm:ssZ。注意：后付费模式本项为null",
"example": "2020-09-22T00:00:00+00:00"
},
{
"name": "OsName",
"type": "文本",
"desc": "操作系统名称。",
"example": "CentOS 7.4 64bit"
},
{
"name": "SecurityGroupIds",
"type": "文本、多值",
"desc": "实例所属安全组。该参数可以通过调用 DescribeSecurityGroups 的返回值中的sgId字段来获取。",
"example": [
"sg-p1ezv4wz"
]
},
{
"name": "LoginSettings",
"type": "json",
"desc": "实例登录设置。目前只返回实例所关联的密钥。",
"example": {
"Password": "123qwe!@#QWE",
"KeepImageLogin": "False",
"KeyIds": [
"skey-b4vakk62"
]
}
},
{
"name": "InstanceState",
"type": "文本",
"desc": "实例状态。取值范围： PENDING：表示创建中 LAUNCH_FAILED：表示创建失败 RUNNING：表示运行中 STOPPED：表示关机 STARTING：表示开机中 STOPPING：表示关机中 REBOOTING：表示重启中 SHUTDOWN：表示停止待销毁 TERMINATING：表示销毁中。",
"example": "RUNNING"
},
{
"name": "Tags",
"type": "json",
"desc": "实例关联的标签列表。",
"example": [
{
"Value": "test",
"Key": "test"
}
]
},
{
"name": "StopChargingMode",
"type": "文本",
"desc": "实例的关机计费模式。 取值范围： KEEP_CHARGING：关机继续收费 STOP_CHARGING：关机停止收费NOT_APPLICABLE：实例处于非关机状态或者不适用关机停止计费的条件",
"example": "NOT_APPLICABLE"
},
{
"name": "Uuid",
"type": "文本",
"desc": "实例全局唯一ID",
"example": "e85f1388-0422-410d-8e50-bef540e78c18"
},
{
"name": "LatestOperation",
"type": "文本",
"desc": "实例的最新操作。例：StopInstances、ResetInstance。 注意：此字段可能返回 null，表示取不到有效值。",
"example": "ResetInstancesType"
},
{
"name": "LatestOperationState",
"type": "文本",
"desc": "实例的最新操作状态。取值范围： SUCCESS：表示操作成功 OPERATING：表示操作执行中 FAILED：表示操作失败 注意：此字段可能返回 null，表示取不到有效值。",
"example": "SUCCESS"
},
{
"name": "LatestOperationRequestId",
"type": "文本",
"desc": "实例最新操作的唯一请求 ID。 注意：此字段可能返回 null，表示取不到有效值。",
"example": "c7de1287-061d-4ace-8caf-6ad8e5a2f29a"
},
{
"name": "DisasterRecoverGroupId",
"type": "文本",
"desc": "分散置放群组ID。 注意：此字段可能返回 null，表示取不到有效值。",
"example": ""
},
{
"name": "IPv6Addresses",
"type": "文本、多值",
"desc": "实例的IPv6地址。 注意：此字段可能返回 null，表示取不到有效值。",
"example": [
"2001:0db8:86a3:08d3:1319:8a2e:0370:7344"
]
},
{
"name": "CamRoleName",
"type": "文本",
"desc": "CAM角色名。 注意：此字段可能返回 null，表示取不到有效值。",
"example": ""
},
{
"name": "HpcClusterId",
"type": "文本",
"desc": "高性能计算集群ID。 注意：此字段可能返回 null，表示取不到有效值。",
"example": ""
},
{
"name": "RdmaIpAddresses",
"type": "文本、多值",
"desc": "高性能计算集群IP列表。 注意：此字段可能返回 null，表示取不到有效值。",
"example": []
},
{
"name": "IsolatedSource",
"type": "文本",
"desc": "实例隔离类型。取值范围： ARREAR：表示欠费隔离 EXPIRE：表示到期隔离 MANMADE：表示主动退还隔离 NOTISOLATED：表示未隔离 注意：此字段可能返回 null，表示取不到有效值。",
"example": "NOTISOLATED"
},
{
"name": "GPUInfo",
"type": "json",
"desc": "GPU信息。如果是gpu类型子机，该值会返回GPU信息，如果是其他类型子机则不返回。 注意：此字段可能返回 null，表示取不到有效值。",
"example": null
},
{
"name": "LicenseType",
"type": "文本",
"desc": "实例的操作系统许可类型，默认为TencentCloud",
"example": null
},
{
"name": "DisableApiTermination",
"type": "Boolean",
"desc": "实例销毁保护标志，表示是否允许通过api接口删除实例。取值范围： TRUE：表示开启实例保护，不允许通过api接口删除实例 FALSE：表示关闭实例保护，允许通过api接口删除实例 默认取值：FALSE。",
"example": null
},
{
"name": "DefaultLoginUser",
"type": "文本",
"desc": "默认登录用户。",
"example": null
},
{
"name": "DefaultLoginPort",
"type": "整数",
"desc": "默认登录端口。",
"example": null
},
{
"name": "LatestOperationErrorMsg",
"type": "文本",
"desc": "实例的最新操作错误信息。 注意：此字段可能返回 null，表示取不到有效值。",
"example": null
}
]

140
cmdb-api/api/lib/cmdb/cache.py
View File

@ -2,8 +2,14 @@
from __future__ import unicode_literals
import requests
from flask import current_app
from api.extensions import cache
from api.extensions import db
from api.lib.cmdb.custom_dashboard import CustomDashboardManager
from api.models.cmdb import Attribute
from api.models.cmdb import CI
from api.models.cmdb import CIType
from api.models.cmdb import CITypeAttribute
from api.models.cmdb import RelationType
@ -115,6 +121,9 @@ class CITypeAttributesCache(object):
PREFIX_ID = "CITypeAttributes::TypeID::{0}"
PREFIX_NAME = "CITypeAttributes::TypeName::{0}"
PREFIX_ID2 = "CITypeAttributes2::TypeID::{0}"
PREFIX_NAME2 = "CITypeAttributes2::TypeName::{0}"
@classmethod
def get(cls, key):
if key is None:
@ -132,6 +141,29 @@ class CITypeAttributesCache(object):
cls.set(key, attrs)
return attrs
@classmethod
def get2(cls, key):
"""
return [(type_attr, attr), ]
:param key:
:return:
"""
if key is None:
return
attrs = cache.get(cls.PREFIX_NAME2.format(key))
attrs = attrs or cache.get(cls.PREFIX_ID2.format(key))
if not attrs:
attrs = CITypeAttribute.get_by(type_id=key, to_dict=False)
if not attrs:
ci_type = CIType.get_by(name=key, first=True, to_dict=False)
if ci_type is not None:
attrs = CITypeAttribute.get_by(type_id=ci_type.id, to_dict=False)
if attrs is not None:
attrs = [(i, AttributeCache.get(i.attr_id)) for i in attrs]
cls.set2(key, attrs)
return attrs
@classmethod
def set(cls, key, values):
ci_type = CITypeCache.get(key)
@ -139,6 +171,13 @@ class CITypeAttributesCache(object):
cache.set(cls.PREFIX_ID.format(ci_type.id), values)
cache.set(cls.PREFIX_NAME.format(ci_type.name), values)
@classmethod
def set2(cls, key, values):
ci_type = CITypeCache.get(key)
if ci_type is not None:
cache.set(cls.PREFIX_ID2.format(ci_type.id), values)
cache.set(cls.PREFIX_NAME2.format(ci_type.name), values)
@classmethod
def clean(cls, key):
ci_type = CITypeCache.get(key)
@ -147,6 +186,11 @@ class CITypeAttributesCache(object):
cache.delete(cls.PREFIX_ID.format(ci_type.id))
cache.delete(cls.PREFIX_NAME.format(ci_type.name))
attrs2 = cls.get2(key)
if attrs2 is not None and ci_type:
cache.delete(cls.PREFIX_ID2.format(ci_type.id))
cache.delete(cls.PREFIX_NAME2.format(ci_type.name))
class CITypeAttributeCache(object):
"""
@ -173,3 +217,99 @@ class CITypeAttributeCache(object):
@classmethod
def clean(cls, type_id, attr_id):
cache.delete(cls.PREFIX_ID.format(type_id, attr_id))
class CMDBCounterCache(object):
KEY = 'CMDB::Counter'
@classmethod
def get(cls):
result = cache.get(cls.KEY) or {}
if not result:
result = cls.reset()
return result
@classmethod
def set(cls, result):
cache.set(cls.KEY, result, timeout=0)
@classmethod
def reset(cls):
customs = CustomDashboardManager.get()
result = {}
for custom in customs:
if custom['category'] == 0:
result[custom['id']] = cls.summary_counter(custom['type_id'])
elif custom['category'] == 1:
result[custom['id']] = cls.attribute_counter(custom['type_id'], custom['attr_id'])
elif custom['category'] == 2:
result[custom['id']] = cls.relation_counter(custom['type_id'], custom['level'])
cls.set(result)
return result
@classmethod
def update(cls, custom):
result = cache.get(cls.KEY) or {}
if not result:
result = cls.reset()
if custom['category'] == 0:
result[custom['id']] = cls.summary_counter(custom['type_id'])
elif custom['category'] == 1:
result[custom['id']] = cls.attribute_counter(custom['type_id'], custom['attr_id'])
elif custom['category'] == 2:
result[custom['id']] = cls.relation_counter(custom['type_id'], custom['level'])
cls.set(result)
@staticmethod
def summary_counter(type_id):
return db.session.query(CI.id).filter(CI.deleted.is_(False)).filter(CI.type_id == type_id).count()
@staticmethod
def relation_counter(type_id, level):
uri = current_app.config.get('CMDB_API')
type_names = requests.get("{}/ci/s?q=_type:{}&count=10000".format(uri, type_id)).json().get('result')
type_id_names = [(str(i.get('_id')), i.get(i.get('unique'))) for i in type_names]
url = "{}/ci_relations/statistics?root_ids={}&level={}".format(
uri, ','.join([i[0] for i in type_id_names]), level)
stats = requests.get(url).json()
id2name = dict(type_id_names)
type_ids = set()
for i in (stats.get('detail') or []):
for j in stats['detail'][i]:
type_ids.add(j)
for type_id in type_ids:
_type = CITypeCache.get(type_id)
id2name[type_id] = _type and _type.alias
result = dict(summary={}, detail={})
for i in stats:
if i == "detail":
for j in stats['detail']:
if id2name[j]:
result['detail'][id2name[j]] = stats['detail'][j]
result['detail'][id2name[j]] = dict()
for _j in stats['detail'][j]:
result['detail'][id2name[j]][id2name[_j]] = stats['detail'][j][_j]
elif id2name.get(i):
result['summary'][id2name[i]] = stats[i]
return result
@staticmethod
def attribute_counter(type_id, attr_id):
uri = current_app.config.get('CMDB_API')
url = "{}/ci/s?q=_type:{}&fl={}&facet={}".format(uri, type_id, attr_id, attr_id)
res = requests.get(url).json()
if res.get('facet'):
return dict([i[:2] for i in list(res.get('facet').values())[0]])

525
cmdb-api/api/lib/cmdb/ci.py
View File

@ -1,32 +1,44 @@
# -*- coding:utf-8 -*-
# -*- coding:utf-8 -*-
import copy
import datetime
import json
from flask import abort
from flask import current_app
from flask import g
from werkzeug.exceptions import BadRequest
from api.extensions import db
from api.extensions import rd
from api.lib.cmdb.cache import AttributeCache
from api.lib.cmdb.cache import CITypeAttributesCache
from api.lib.cmdb.cache import CITypeCache
from api.lib.cmdb.ci_type import CITypeAttributeManager
from api.lib.cmdb.ci_type import CITypeManager
from api.lib.cmdb.ci_type import CITypeRelationManager
from api.lib.cmdb.ci_type import CITypeUniqueConstraintManager
from api.lib.cmdb.const import AttributeDefaultValueEnum
from api.lib.cmdb.const import CMDB_QUEUE
from api.lib.cmdb.const import ConstraintEnum
from api.lib.cmdb.const import ExistPolicy
from api.lib.cmdb.const import OperateType
from api.lib.cmdb.const import REDIS_PREFIX_CI
from api.lib.cmdb.const import ResourceTypeEnum, PermEnum
from api.lib.cmdb.const import RetKey
from api.lib.cmdb.history import AttributeHistoryManger
from api.lib.cmdb.history import CIRelationHistoryManager
from api.lib.cmdb.search.ci.db.query_sql import QUERY_CIS_BY_IDS
from api.lib.cmdb.search.ci.db.query_sql import QUERY_CIS_BY_VALUE_TABLE
from api.lib.cmdb.perms import CIFilterPermsCRUD
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.cmdb.utils import TableMap
from api.lib.cmdb.utils import ValueTypeMap
from api.lib.cmdb.value import AttributeValueManager
from api.lib.decorator import kwargs_required
from api.lib.perm.acl.acl import ACLManager
from api.lib.perm.acl.acl import is_app_admin
from api.lib.perm.acl.acl import validate_permission
from api.lib.utils import Lock
from api.lib.utils import handle_arg_list
from api.models.cmdb import CI
from api.models.cmdb import CIRelation
@ -45,27 +57,36 @@ class CIManager(object):
def __init__(self):
pass
@staticmethod
def get_by_id(ci_id):
return CI.get_by_id(ci_id)
@staticmethod
def get_type_name(ci_id):
ci = CI.get_by_id(ci_id) or abort(404, "CI <{0}> is not existed".format(ci_id))
ci = CI.get_by_id(ci_id) or abort(404, ErrFormat.ci_not_found.format("id={}".format(ci_id)))
return CITypeCache.get(ci.type_id).name
@staticmethod
def get_type(ci_id):
ci = CI.get_by_id(ci_id) or abort(404, ErrFormat.ci_not_found.format("id={}".format(ci_id)))
return CITypeCache.get(ci.type_id)
@staticmethod
def confirm_ci_existed(ci_id):
return CI.get_by_id(ci_id) or abort(404, "CI <{0}> is not existed".format(ci_id))
return CI.get_by_id(ci_id) or abort(404, ErrFormat.ci_not_found.format("id={}".format(ci_id)))
@classmethod
def get_ci_by_id(cls, ci_id, ret_key=RetKey.NAME, fields=None, need_children=True):
"""
:param ci_id:
:param ci_id:
:param ret_key: name, id, or alias
:param fields: attribute list
:param need_children:
:return:
:param need_children:
:return:
"""
ci = CI.get_by_id(ci_id) or abort(404, "CI <{0}> is not existed".format(ci_id))
ci = CI.get_by_id(ci_id) or abort(404, ErrFormat.ci_not_found.format("id={}".format(ci_id)))
res = dict()
@ -83,19 +104,63 @@ class CIManager(object):
return res
@staticmethod
def get_ci_by_id_from_db(ci_id, ret_key=RetKey.NAME, fields=None, need_children=True, use_master=False):
"""
:param ci_id:
:param ret_key: name, id or alias
:param fields: list
:param need_children:
:param use_master: whether to use master db
:return:
@classmethod
def valid_ci_only_read(cls, ci):
if is_app_admin("cmdb"):
return
validate_permission(CIManager.get_type_name(ci.id), ResourceTypeEnum.CI, PermEnum.READ, "cmdb")
acl = ACLManager('cmdb')
res = acl.get_resources(ResourceTypeEnum.CI_FILTER)
if res and ci.type_id in CIFilterPermsCRUD().get_by_ids(list(map(int, [i['name'] for i in res]))):
return abort(403, ErrFormat.no_permission2)
@classmethod
def _valid_ci_for_no_read(cls, ci, ci_type=None):
type_id = ci.type_id if ci else ci_type.id
acl = ACLManager('cmdb')
res = acl.get_resources(ResourceTypeEnum.CI_FILTER)
type2filters = CIFilterPermsCRUD().get_by_ids(list(map(int, [i['name'] for i in res])), type_id=type_id)
if res and type_id in type2filters:
if type2filters[type_id].get('ci_filter') and ci:
from api.lib.cmdb.search import SearchError
from api.lib.cmdb.search.ci import search
query = "_id:{},{}".format(ci.id, type2filters[type_id].get('ci_filter'))
s = search(query)
try:
response, _, _, _, _, _ = s.search()
except SearchError as e:
current_app.logger.warning(e)
return abort(400, str(e))
if not response:
return abort(403, ErrFormat.ci_filter_perm_ci_no_permission)
return type2filters[type_id].get('attr_filter') or []
@classmethod
def get_ci_by_id_from_db(cls, ci_id, ret_key=RetKey.NAME, fields=None, need_children=True, use_master=False,
valid=False):
"""
ci = CI.get_by_id(ci_id) or abort(404, "CI <{0}> is not existed".format(ci_id))
:param ci_id:
:param ret_key: name, id or alias
:param fields: list
:param need_children:
:param use_master: whether to use master db
:param valid:
:return:
"""
ci = CI.get_by_id(ci_id) or abort(404, ErrFormat.ci_not_found.format("id={}".format(ci_id)))
if valid:
cls.valid_ci_only_read(ci)
res = dict()
@ -115,8 +180,9 @@ class CIManager(object):
use_master=use_master)
res.update(_res)
res['type_id'] = ci_type.id
res['ci_id'] = ci_id
res['_type'] = ci_type.id
res['ci_type_alias'] = ci_type.alias
res['_id'] = ci_id
return res
@ -134,19 +200,33 @@ class CIManager(object):
return numfound, page, res
@classmethod
def get_ad_statistics(cls):
res = CI.get_by(to_dict=False)
result = dict()
for i in res:
result.setdefault(i.type_id, dict(total=0, auto_discovery=0))
result[i.type_id]['total'] += 1
if i.is_auto_discovery:
result[i.type_id]['auto_discovery'] += 1
return result
@staticmethod
def ci_is_exist(unique_key, unique_value):
def ci_is_exist(unique_key, unique_value, type_id):
"""
:param unique_key: is a attribute
:param unique_value:
:return:
:param unique_value:
:param type_id:
:return:
"""
value_table = TableMap(attr_name=unique_key.name).table
unique = value_table.get_by(attr_id=unique_key.id,
value=unique_value,
to_dict=False,
first=True)
value_table = TableMap(attr=unique_key).table
unique = db.session.query(value_table).join(CI, CI.id == value_table.ci_id).filter(
value_table.attr_id == unique_key.id).filter(value_table.value == unique_value).filter(
CI.type_id == type_id).filter(CI.deleted.is_(False)).filter(value_table.deleted.is_(False)).first()
if unique:
return CI.get_by_id(unique.ci_id)
@ -155,82 +235,218 @@ class CIManager(object):
ci = CI.get_by_id(ci_id)
ci.delete() # TODO: soft delete
@staticmethod
def _valid_unique_constraint(type_id, ci_dict, ci_id=None):
unique_constraints = CITypeUniqueConstraintManager.get_by_type_id(type_id)
if not unique_constraints:
return
attr_ids = []
for i in unique_constraints:
attr_ids.extend(i.attr_ids)
attrs = [AttributeCache.get(i) for i in list(set(attr_ids))]
id2name = {i.id: i.name for i in attrs if i}
not_existed_fields = list(set(id2name.values()) - set(ci_dict.keys()))
if not_existed_fields and ci_id is not None:
ci_dict = copy.deepcopy(ci_dict)
ci_dict.update(AttributeValueManager().get_attr_values(not_existed_fields, ci_id))
for constraint in unique_constraints:
ci_ids = None
for attr_id in constraint.attr_ids:
value_table = TableMap(attr_name=id2name[attr_id]).table
_ci_ids = set([i.ci_id for i in value_table.get_by(attr_id=attr_id,
to_dict=False,
value=ci_dict.get(id2name[attr_id]) or None)])
if ci_ids is None:
ci_ids = _ci_ids
else:
ci_ids &= _ci_ids
if ci_ids - (ci_id and {ci_id} or set()):
return abort(400, ErrFormat.unique_constraint.format(
" - ".join([id2name[i] for i in constraint.attr_ids])))
@staticmethod
def _auto_inc_id(attr):
db.session.remove()
value_table = TableMap(attr_name=attr.name).table
with Lock("auto_inc_id_{}".format(attr.name), need_lock=True):
max_v = value_table.get_by(attr_id=attr.id, only_query=True).order_by(
getattr(value_table, 'value').desc()).first()
if max_v is not None:
return int(max_v.value) + 1
return 1
@classmethod
def add(cls, ci_type_name, exist_policy=ExistPolicy.REPLACE, _no_attribute_policy=ExistPolicy.IGNORE, **ci_dict):
def add(cls, ci_type_name,
exist_policy=ExistPolicy.REPLACE,
_no_attribute_policy=ExistPolicy.IGNORE,
is_auto_discovery=False,
_is_admin=False,
**ci_dict):
"""
:param ci_type_name:
:param ci_type_name:
:param exist_policy: replace or reject or need
:param _no_attribute_policy: ignore or reject
:param ci_dict:
:return:
:param is_auto_discovery: default is False
:param _is_admin: default is False
:param ci_dict:
:return:
"""
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
ci_type = CITypeManager.check_is_existed(ci_type_name)
unique_key = AttributeCache.get(ci_type.unique_id) or abort(400, 'illegality unique attribute')
unique_key = AttributeCache.get(ci_type.unique_id) or abort(
400, ErrFormat.unique_value_not_found.format("unique_id={}".format(ci_type.unique_id)))
unique_value = ci_dict.get(unique_key.name)
unique_value = unique_value or ci_dict.get(unique_key.alias)
unique_value = unique_value or ci_dict.get(unique_key.id)
unique_value = unique_value or abort(400, '{0} missing'.format(unique_key.name))
unique_value = unique_value or abort(400, ErrFormat.unique_key_required.format(unique_key.name))
existed = cls.ci_is_exist(unique_key, unique_value)
if existed is not None:
if exist_policy == ExistPolicy.REJECT:
return abort(400, 'CI is already existed')
if existed.type_id != ci_type.id:
existed.update(type_id=ci_type.id)
ci = existed
else:
if exist_policy == ExistPolicy.NEED:
return abort(404, 'CI <{0}> does not exist'.format(unique_value))
ci = CI.create(type_id=ci_type.id)
attrs = CITypeAttributesCache.get2(ci_type_name)
ci_type_attrs_name = {attr.name: attr for _, attr in attrs}
ci_type_attrs_alias = {attr.alias: attr for _, attr in attrs}
ci_attr2type_attr = {type_attr.attr_id: type_attr for type_attr, _ in attrs}
ci = None
need_lock = g.user.username not in ("worker", "cmdb_agent", "agent")
with Lock(ci_type_name, need_lock=need_lock):
existed = cls.ci_is_exist(unique_key, unique_value, ci_type.id)
if existed is not None:
if exist_policy == ExistPolicy.REJECT:
return abort(400, ErrFormat.ci_is_already_existed)
if existed.type_id != ci_type.id:
existed.update(type_id=ci_type.id)
ci = existed
else:
if exist_policy == ExistPolicy.NEED:
return abort(404, ErrFormat.ci_not_found.format("{}={}".format(unique_key.name, unique_value)))
from api.lib.cmdb.const import L_CI
if L_CI and len(CI.get_by(type_id=ci_type.id)) > L_CI * 2:
return abort(400, ErrFormat.limit_ci.format(L_CI * 2))
limit_attrs = cls._valid_ci_for_no_read(ci, ci_type) if not _is_admin else {}
if existed is None: # set default
for type_attr, attr in attrs:
if attr.default and attr.default.get('default') is not None:
if attr.default.get('default') and attr.default.get('default') in (
AttributeDefaultValueEnum.CREATED_AT, AttributeDefaultValueEnum.UPDATED_AT):
ci_dict[attr.name] = now
elif attr.default.get('default') == AttributeDefaultValueEnum.AUTO_INC_ID:
ci_dict[attr.name] = cls._auto_inc_id(attr)
elif ((attr.name not in ci_dict and attr.alias not in ci_dict) or (
ci_dict.get(attr.name) is None and ci_dict.get(attr.alias) is None)):
ci_dict[attr.name] = attr.default.get('default')
if type_attr.is_required and (attr.name not in ci_dict and attr.alias not in ci_dict):
return abort(400, ErrFormat.attribute_value_required.format(attr.name))
else:
for type_attr, attr in attrs:
if attr.default and attr.default.get('default') == AttributeDefaultValueEnum.UPDATED_AT:
ci_dict[attr.name] = now
computed_attrs = [attr.to_dict() for _, attr in attrs if attr.is_computed] or None
value_manager = AttributeValueManager()
if computed_attrs:
value_manager.handle_ci_compute_attributes(ci_dict, computed_attrs, ci)
cls._valid_unique_constraint(ci_type.id, ci_dict, ci and ci.id)
for k in ci_dict:
if k not in ci_type_attrs_name and k not in ci_type_attrs_alias and \
_no_attribute_policy == ExistPolicy.REJECT:
return abort(400, ErrFormat.attribute_not_found.format(k))
if limit_attrs and ci_type_attrs_name.get(k) not in limit_attrs and \
ci_type_attrs_alias.get(k) not in limit_attrs:
return abort(403, ErrFormat.ci_filter_perm_attr_no_permission.format(k))
ci_dict = {k: v for k, v in ci_dict.items() if k in ci_type_attrs_name or k in ci_type_attrs_alias}
key2attr = value_manager.valid_attr_value(ci_dict, ci_type.id, ci and ci.id,
ci_type_attrs_name, ci_type_attrs_alias, ci_attr2type_attr)
ci_type_attrs_name = [attr["name"] for attr in CITypeAttributeManager().get_attributes_by_type_id(ci_type.id)]
value_manager = AttributeValueManager()
for p, v in ci_dict.items():
if p not in ci_type_attrs_name:
current_app.logger.warning('ci_type: {0} not has attribute {1}, please check!'.format(ci_type_name, p))
continue
try:
value_manager.create_or_update_attr_value(p, v, ci, _no_attribute_policy)
ci = ci or CI.create(type_id=ci_type.id, is_auto_discovery=is_auto_discovery)
record_id = value_manager.create_or_update_attr_value2(ci, ci_dict, key2attr)
except BadRequest as e:
if existed is None:
cls.delete(ci.id)
raise e
ci_cache.apply_async([ci.id], queue=CMDB_QUEUE)
if record_id: # has change
ci_cache.apply_async([ci.id], queue=CMDB_QUEUE)
return ci.id
def update(self, ci_id, **ci_dict):
def update(self, ci_id, _is_admin=False, **ci_dict):
now = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
ci = self.confirm_ci_existed(ci_id)
ci_type_attrs_name = [attr["name"] for attr in CITypeAttributeManager().get_attributes_by_type_id(ci.type_id)]
attrs = CITypeAttributesCache.get2(ci.type_id)
ci_type_attrs_name = {attr.name: attr for _, attr in attrs}
ci_attr2type_attr = {type_attr.attr_id: type_attr for type_attr, _ in attrs}
for _, attr in attrs:
if attr.default and attr.default.get('default') == AttributeDefaultValueEnum.UPDATED_AT:
ci_dict[attr.name] = now
computed_attrs = [attr.to_dict() for _, attr in attrs if attr.is_computed] or None
value_manager = AttributeValueManager()
for p, v in ci_dict.items():
if p not in ci_type_attrs_name:
current_app.logger.warning('ci_type: {0} not has attribute {1}, please check!'.format(ci.type_id, p))
continue
if computed_attrs:
value_manager.handle_ci_compute_attributes(ci_dict, computed_attrs, ci)
limit_attrs = self._valid_ci_for_no_read(ci) if not _is_admin else {}
need_lock = g.user.username not in ("worker", "cmdb_agent", "agent")
with Lock(ci.ci_type.name, need_lock=need_lock):
self._valid_unique_constraint(ci.type_id, ci_dict, ci_id)
ci_dict = {k: v for k, v in ci_dict.items() if k in ci_type_attrs_name}
key2attr = value_manager.valid_attr_value(ci_dict, ci.type_id, ci.id, ci_type_attrs_name,
ci_attr2type_attr=ci_attr2type_attr)
if limit_attrs:
for k in ci_dict:
if k not in limit_attrs:
return abort(403, ErrFormat.ci_filter_perm_attr_no_permission.format(k))
try:
value_manager.create_or_update_attr_value(p, v, ci)
record_id = value_manager.create_or_update_attr_value2(ci, ci_dict, key2attr)
except BadRequest as e:
raise e
ci_cache.apply_async([ci_id], queue=CMDB_QUEUE)
if record_id: # has change
ci_cache.apply_async([ci_id], queue=CMDB_QUEUE)
@staticmethod
def update_unique_value(ci_id, unique_name, unique_value):
ci = CI.get_by_id(ci_id) or abort(404, "CI <{0}> is not found".format(ci_id))
ci = CI.get_by_id(ci_id) or abort(404, ErrFormat.ci_not_found.format("id={}".format(ci_id)))
AttributeValueManager().create_or_update_attr_value(unique_name, unique_value, ci)
ci_cache.apply_async([ci_id], queue=CMDB_QUEUE)
@staticmethod
def delete(ci_id):
ci = CI.get_by_id(ci_id) or abort(404, "CI <{0}> is not found".format(ci_id))
@classmethod
def delete(cls, ci_id):
ci = CI.get_by_id(ci_id) or abort(404, ErrFormat.ci_not_found.format("id={}".format(ci_id)))
cls._valid_ci_for_no_read(ci)
ci_dict = cls.get_cis_by_ids([ci_id])
ci_dict = ci_dict and ci_dict[0]
attrs = CITypeAttribute.get_by(type_id=ci.type_id, to_dict=False)
attr_names = set([AttributeCache.get(attr.attr_id).name for attr in attrs])
@ -249,7 +465,7 @@ class CIManager(object):
ci.delete() # TODO: soft delete
AttributeHistoryManger.add(ci_id, [(None, OperateType.DELETE, None, None)])
AttributeHistoryManger.add(None, ci_id, [(None, OperateType.DELETE, ci_dict, None)], ci.type_id)
ci_delete.apply_async([ci.id], queue=CMDB_QUEUE)
@ -260,15 +476,15 @@ class CIManager(object):
ci_type = CITypeManager().check_is_existed(ci_type)
unique_key = AttributeCache.get(ci_type.unique_id)
value_table = TableMap(attr_name=unique_key.name).table
value_table = TableMap(attr=unique_key).table
v = value_table.get_by(attr_id=unique_key.id,
value=unique_value,
to_dict=False,
first=True) \
or abort(404, "not found")
or abort(404, ErrFormat.not_found)
ci = CI.get_by_id(v.ci_id) or abort(404, "CI <{0}> is not found".format(v.ci_id))
ci = CI.get_by_id(v.ci_id) or abort(404, ErrFormat.ci_not_found.format("id={}".format(v.ci_id)))
ci.update(heartbeat=datetime.datetime.now())
@ -315,25 +531,38 @@ class CIManager(object):
return numfound, result
@staticmethod
def _get_cis_from_cache(ci_ids, ret_key=RetKey.NAME, fields=None):
def _get_cis_from_cache(ci_ids, ret_key=RetKey.NAME, fields=None, unique_required=False, excludes=None):
res = rd.get(ci_ids, REDIS_PREFIX_CI)
if res is not None and None not in res and ret_key == RetKey.NAME:
res = list(map(json.loads, res))
if not fields:
if not fields and not excludes:
return res
else:
elif fields:
_res = []
for d in res:
_d = dict()
_d["_id"], _d["_type"] = d.get("_id"), d.get("_type")
_d["ci_type"] = d.get("ci_type")
for field in fields:
if unique_required:
_d[d.get('unique')] = d.get(d.get('unique'))
for field in fields + ['ci_type_alias', 'unique', 'unique_alias']:
_d[field] = d.get(field)
_res.append(_d)
return _res
else:
excludes = set(excludes)
for i in res:
for e in excludes:
i.pop(e, None)
return res
@staticmethod
def _get_cis_from_db(ci_ids, ret_key=RetKey.NAME, fields=None, value_tables=None):
def _get_cis_from_db(ci_ids, ret_key=RetKey.NAME, fields=None, value_tables=None, excludes=None):
from api.lib.cmdb.search.ci.db.query_sql import QUERY_CIS_BY_IDS
from api.lib.cmdb.search.ci.db.query_sql import QUERY_CIS_BY_VALUE_TABLE
if not fields:
filter_fields_sql = ""
else:
@ -344,7 +573,7 @@ class CIManager(object):
_fields.append(str(attr.id))
filter_fields_sql = "WHERE A.attr_id in ({0})".format(",".join(_fields))
ci_ids = ",".join(ci_ids)
ci_ids = ",".join(map(str, ci_ids))
if value_tables is None:
value_tables = ValueTypeMap.table_name.values()
@ -356,14 +585,23 @@ class CIManager(object):
ci_set = set()
res = list()
ci_dict = dict()
unique_id2obj = dict()
excludes = excludes and set(excludes)
for ci_id, type_id, attr_id, attr_name, attr_alias, value, value_type, is_list in cis:
if not fields and excludes and (attr_name in excludes or attr_alias in excludes):
continue
if ci_id not in ci_set:
ci_dict = dict()
ci_type = CITypeCache.get(type_id)
ci_dict["ci_id"] = ci_id
ci_dict["ci_type"] = type_id
ci_dict["_id"] = ci_id
ci_dict["_type"] = type_id
ci_dict["ci_type"] = ci_type.name
ci_dict["ci_type_alias"] = ci_type.alias
if ci_type.unique_id not in unique_id2obj:
unique_id2obj[ci_type.unique_id] = AttributeCache.get(ci_type.unique_id)
ci_dict["unique"] = unique_id2obj[ci_type.unique_id] and unique_id2obj[ci_type.unique_id].name
ci_dict["unique_alias"] = unique_id2obj[ci_type.unique_id] and unique_id2obj[ci_type.unique_id].alias
ci_set.add(ci_id)
res.append(ci_dict)
@ -374,7 +612,7 @@ class CIManager(object):
elif ret_key == RetKey.ID:
attr_key = attr_id
else:
return abort(400, "invalid ret key")
return abort(400, ErrFormat.argument_invalid.format("ret_key"))
value = ValueTypeMap.serialize2[value_type](value)
if is_list:
@ -385,14 +623,17 @@ class CIManager(object):
return res
@classmethod
def get_cis_by_ids(cls, ci_ids, ret_key=RetKey.NAME, fields=None, value_tables=None):
def get_cis_by_ids(cls, ci_ids, ret_key=RetKey.NAME,
fields=None, value_tables=None, unique_required=False, excludes=None):
"""
:param ci_ids: list of CI instance ID, eg. ['1', '2']
:param ret_key: name, id or alias
:param fields:
:param value_tables:
:return:
:param fields:
:param value_tables:
:param unique_required:
:param excludes: exclude field list
:return:
"""
if not ci_ids:
@ -401,12 +642,12 @@ class CIManager(object):
fields = [] if fields is None or not isinstance(fields, list) else fields
ci_id_tuple = tuple(map(int, ci_ids))
res = cls._get_cis_from_cache(ci_id_tuple, ret_key, fields)
res = cls._get_cis_from_cache(ci_id_tuple, ret_key, fields, unique_required, excludes=excludes)
if res is not None:
return res
current_app.logger.warning("cache not hit...............")
return cls._get_cis_from_db(ci_ids, ret_key, fields, value_tables)
return cls._get_cis_from_db(ci_ids, ret_key, fields, value_tables, excludes=excludes)
class CIRelationManager(object):
@ -493,7 +734,29 @@ class CIRelationManager(object):
return numfound, len(first_ci_ids), result
@classmethod
def add(cls, first_ci_id, second_ci_id, more=None, relation_type_id=None, many_to_one=False):
def get_ancestor_ids(cls, ci_ids, level=1):
for _ in range(level):
cis = db.session.query(CIRelation.first_ci_id).filter(
CIRelation.second_ci_id.in_(ci_ids)).filter(CIRelation.deleted.is_(False))
ci_ids = [i.first_ci_id for i in cis]
return ci_ids
@staticmethod
def _check_constraint(first_ci_id, second_ci_id, type_relation):
if type_relation.constraint == ConstraintEnum.Many2Many:
return
first_existed = CIRelation.get_by(first_ci_id=first_ci_id, relation_type_id=type_relation.relation_type_id)
second_existed = CIRelation.get_by(second_ci_id=second_ci_id, relation_type_id=type_relation.relation_type_id)
if type_relation.constraint == ConstraintEnum.One2One and (first_existed or second_existed):
return abort(400, ErrFormat.relation_constraint.format("1对1"))
if type_relation.constraint == ConstraintEnum.One2Many and second_existed:
return abort(400, ErrFormat.relation_constraint.format("1对多"))
@classmethod
def add(cls, first_ci_id, second_ci_id, more=None, relation_type_id=None):
first_ci = CIManager.confirm_ci_existed(first_ci_id)
second_ci = CIManager.confirm_ci_existed(second_ci_id)
@ -514,18 +777,22 @@ class CIRelationManager(object):
first=True,
to_dict=False)
relation_type_id = type_relation and type_relation.relation_type_id
relation_type_id or abort(404, "Relation {0} <-> {1} is not found".format(
first_ci.ci_type.name, second_ci.ci_type.name))
relation_type_id or abort(404, ErrFormat.relation_not_found.format("{} -> {}".format(
first_ci.ci_type.name, second_ci.ci_type.name)))
if many_to_one:
for item in CIRelation.get_by(second_ci_id=second_ci_id,
relation_type_id=relation_type_id,
to_dict=False):
item.soft_delete()
his_manager = CIRelationHistoryManager()
his_manager.add(item, operate_type=OperateType.DELETE)
if current_app.config.get('USE_ACL'):
resource_name = CITypeRelationManager.acl_resource_name(first_ci.ci_type.name,
second_ci.ci_type.name)
if not ACLManager().has_permission(
resource_name,
ResourceTypeEnum.CI_TYPE_RELATION,
PermEnum.ADD):
return abort(403, ErrFormat.no_permission.format(resource_name, PermEnum.ADD))
ci_relation_delete.apply_async(args=(item.first_ci_id, item.second_ci_id), queue=CMDB_QUEUE)
else:
type_relation = CITypeRelation.get_by_id(relation_type_id)
cls._check_constraint(first_ci_id, second_ci_id, type_relation)
existed = CIRelation.create(first_ci_id=first_ci_id,
second_ci_id=second_ci_id,
@ -542,7 +809,16 @@ class CIRelationManager(object):
@staticmethod
def delete(cr_id):
cr = CIRelation.get_by_id(cr_id) or abort(404, "CIRelation <{0}> is not existed".format(cr_id))
cr = CIRelation.get_by_id(cr_id) or abort(404, ErrFormat.relation_not_found.format("id={}".format(cr_id)))
if current_app.config.get('USE_ACL'):
resource_name = CITypeRelationManager.acl_resource_name(cr.first_ci.ci_type.name, cr.second_ci.ci_type.name)
if not ACLManager().has_permission(
resource_name,
ResourceTypeEnum.CI_TYPE_RELATION,
PermEnum.DELETE):
return abort(403, ErrFormat.no_permission.format(resource_name, PermEnum.DELETE))
cr.delete()
his_manager = CIRelationHistoryManager()
@ -564,23 +840,34 @@ class CIRelationManager(object):
return cls.delete(cr.id)
@classmethod
def batch_update(cls, ci_ids, parents):
def batch_update(cls, ci_ids, parents, children):
"""
only for many to one
:param ci_ids:
:param parents:
:return:
:param ci_ids:
:param parents:
:param children:
:return:
"""
from api.lib.cmdb.utils import TableMap
if parents is not None and isinstance(parents, list):
for parent_id in parents:
for ci_id in ci_ids:
cls.add(parent_id, ci_id)
if parents is not None and isinstance(parents, dict):
for attr_name in parents:
if parents[attr_name]:
attr = AttributeCache.get(attr_name)
value_table = TableMap(attr_name=attr.name).table
parent = value_table.get_by(attr_id=attr.id, value=parents[attr_name], first=True, to_dict=False)
if not parent:
return abort(404, "{0}: {1} is not found".format(attr_name, parents[attr_name]))
parent_id = parent.ci_id
for ci_id in ci_ids:
cls.add(parent_id, ci_id, many_to_one=True)
if children is not None and isinstance(children, list):
for child_id in children:
for ci_id in ci_ids:
cls.add(ci_id, child_id)
@classmethod
def batch_delete(cls, ci_ids, parents):
"""
only for many to one
:param ci_ids:
:param parents:
:return:
"""
if parents is not None and isinstance(parents, list):
for parent_id in parents:
for ci_id in ci_ids:
cls.delete_2(parent_id, ci_id)

729
cmdb-api/api/lib/cmdb/ci_type.py
View File

File diff suppressed because it is too large Load Diff

53
cmdb-api/api/lib/cmdb/const.py
View File

@ -14,6 +14,12 @@ class ValueTypeEnum(BaseEnum):
JSON = "6"
class ConstraintEnum(BaseEnum):
One2Many = "0"
One2One = "1"
Many2Many = "2"
class CIStatusEnum(BaseEnum):
REVIEW = "0"
VALIDATE = "1"
@ -32,6 +38,23 @@ class OperateType(BaseEnum):
UPDATE = "2"
class CITypeOperateType(BaseEnum):
ADD = "0" # 新增模型
UPDATE = "1" # 修改模型
DELETE = "2" # 删除模型
ADD_ATTRIBUTE = "3" # 新增属性
UPDATE_ATTRIBUTE = "4" # 修改属性
DELETE_ATTRIBUTE = "5" # 删除属性
ADD_TRIGGER = "6" # 新增触发器
UPDATE_TRIGGER = "7" # 修改触发器
DELETE_TRIGGER = "8" # 删除触发器
ADD_UNIQUE_CONSTRAINT = "9" # 新增联合唯一
UPDATE_UNIQUE_CONSTRAINT = "10" # 修改联合唯一
DELETE_UNIQUE_CONSTRAINT = "11" # 删除联合唯一
ADD_RELATION = "12" # 新增关系
DELETE_RELATION = "13" # 删除关系
class RetKey(BaseEnum):
ID = "id"
NAME = "name"
@ -40,21 +63,41 @@ class RetKey(BaseEnum):
class ResourceTypeEnum(BaseEnum):
CI = "CIType"
RELATION_VIEW = "RelationView"
CI_TYPE = "CIType" # create/update/delete/read/config/grant
CI_TYPE_RELATION = "CITypeRelation" # create/delete/grant
RELATION_VIEW = "RelationView" # read/update/delete/grant
CI_FILTER = "CIFilter" # read
class PermEnum(BaseEnum):
ADD = "add"
ADD = "create"
UPDATE = "update"
DELETE = "delete"
READ = "read"
CONFIG = "config"
GRANT = "grant"
class RoleEnum(BaseEnum):
CONFIG = "admin"
CONFIG = "cmdb_admin"
CMDB_READ_ALL = "CMDB_READ_ALL"
CMDB_QUEUE = "cmdb_async"
REDIS_PREFIX_CI = "CMDB_CI"
class AutoDiscoveryType(BaseEnum):
AGENT = "agent"
SNMP = "snmp"
HTTP = "http"
class AttributeDefaultValueEnum(BaseEnum):
CREATED_AT = "$created_at"
UPDATED_AT = "$updated_at"
AUTO_INC_ID = "$auto_inc_id"
CMDB_QUEUE = "one_cmdb_async"
REDIS_PREFIX_CI = "ONE_CMDB"
REDIS_PREFIX_CI_RELATION = "CMDB_CI_RELATION"
L_TYPE = None
L_CI = None

74
cmdb-api/api/lib/cmdb/custom_dashboard.py Normal file
View File

@ -0,0 +1,74 @@
# -*- coding:utf-8 -*-
from flask import abort
from api.lib.cmdb.resp_format import ErrFormat
from api.models.cmdb import CustomDashboard
from api.models.cmdb import SystemConfig
class CustomDashboardManager(object):
cls = CustomDashboard
@staticmethod
def get():
return sorted(CustomDashboard.get_by(to_dict=True), key=lambda x: (x["category"], x['order']))
@staticmethod
def add(**kwargs):
from api.lib.cmdb.cache import CMDBCounterCache
if kwargs.get('name'):
CustomDashboard.get_by(name=kwargs['name']) and abort(400, ErrFormat.custom_name_duplicate)
new = CustomDashboard.create(**kwargs)
CMDBCounterCache.update(new.to_dict())
return new
@staticmethod
def update(_id, **kwargs):
from api.lib.cmdb.cache import CMDBCounterCache
existed = CustomDashboard.get_by_id(_id) or abort(404, ErrFormat.not_found)
new = existed.update(**kwargs)
CMDBCounterCache.update(new.to_dict())
return new
@staticmethod
def batch_update(id2options):
for _id in id2options:
existed = CustomDashboard.get_by_id(_id) or abort(404, ErrFormat.not_found)
existed.update(options=id2options[_id])
@staticmethod
def delete(_id):
existed = CustomDashboard.get_by_id(_id) or abort(404, ErrFormat.not_found)
existed.soft_delete()
class SystemConfigManager(object):
cls = SystemConfig
@staticmethod
def get(name):
return SystemConfig.get_by(name=name, first=True, to_dict=True)
@staticmethod
def create_or_update(name, option):
existed = SystemConfig.get_by(name=name, first=True, to_dict=False)
if existed is not None:
return existed.update(option=option)
else:
return SystemConfig.create(name=name, option=option)
@staticmethod
def delete(name):
existed = SystemConfig.get_by(name=name, first=True, to_dict=False) or abort(404, ErrFormat.not_found)
existed.soft_delete()

238
cmdb-api/api/lib/cmdb/history.py
View File

@ -10,76 +10,174 @@ from api.extensions import db
from api.lib.cmdb.cache import AttributeCache
from api.lib.cmdb.cache import RelationTypeCache
from api.lib.cmdb.const import OperateType
from api.lib.cmdb.perms import CIFilterPermsCRUD
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.perm.acl.cache import UserCache
from api.models.cmdb import Attribute
from api.models.cmdb import AttributeHistory
from api.models.cmdb import CIRelationHistory
from api.models.cmdb import CITypeHistory
from api.models.cmdb import CITypeTrigger
from api.models.cmdb import CITypeUniqueConstraint
from api.models.cmdb import OperationRecord
class AttributeHistoryManger(object):
@staticmethod
def get_records(start, end, username, page, page_size):
records = db.session.query(OperationRecord).filter(OperationRecord.deleted.is_(False))
numfound = db.session.query(db.func.count(OperationRecord.id)).filter(OperationRecord.deleted.is_(False))
def get_records_for_attributes(start, end, username, page, page_size, operate_type, type_id,
ci_id=None, attr_id=None):
records = db.session.query(OperationRecord, AttributeHistory).join(
AttributeHistory, OperationRecord.id == AttributeHistory.record_id)
if start:
records = records.filter(OperationRecord.created_at >= start)
numfound = numfound.filter(OperationRecord.created_at >= start)
if end:
records = records.filter(OperationRecord.created_at <= end)
numfound = records.filter(OperationRecord.created_at <= end)
if type_id:
records = records.filter(OperationRecord.type_id == type_id)
if username:
user = UserCache.get(username)
if user:
records = records.filter(OperationRecord.uid == user.uid)
else:
return abort(404, "User <{0}> is not found".format(username))
return abort(404, ErrFormat.user_not_found.format(username))
if operate_type:
records = records.filter(AttributeHistory.operate_type == operate_type)
records = records.order_by(-OperationRecord.id).offset(page_size * (page - 1)).limit(page_size).all()
if ci_id is not None:
records = records.filter(AttributeHistory.ci_id == ci_id)
if attr_id is not None:
records = records.filter(AttributeHistory.attr_id == attr_id)
records = records.order_by(AttributeHistory.id.desc()).offset(page_size * (page - 1)).limit(page_size).all()
total = len(records)
numfound = numfound.first()[0]
res = []
res = {}
for record in records:
_res = record.to_dict()
_res["user"] = UserCache.get(_res.get("uid")).nickname or UserCache.get(_res.get("uid")).username
record_id = record.OperationRecord.id
attr_hist = record.AttributeHistory.to_dict()
attr_hist['attr'] = AttributeCache.get(attr_hist['attr_id'])
if attr_hist['attr']:
attr_hist['attr_name'] = attr_hist['attr'].name
attr_hist['attr_alias'] = attr_hist['attr'].alias
attr_hist.pop("attr")
attr_history = AttributeHistory.get_by(record_id=_res.get("id"), to_dict=False)
_res["attr_history"] = [AttributeCache.get(h.attr_id).attr_alias for h in attr_history]
if record_id not in res:
record_dict = record.OperationRecord.to_dict()
record_dict["user"] = UserCache.get(record_dict.get("uid"))
if record_dict["user"]:
record_dict['user'] = record_dict['user'].nickname
rel_history = CIRelationHistory.get_by(record_id=_res.get("id"), to_dict=False)
rel_statis = {}
for rel in rel_history:
if rel.operate_type not in rel_statis:
rel_statis[rel.operate_type] = 1
else:
rel_statis[rel.operate_type] += 1
_res["rel_history"] = rel_statis
res.append(_res)
res[record_id] = [record_dict, [attr_hist]]
else:
res[record_id][1].append(attr_hist)
return numfound, total, res
attr_filter = CIFilterPermsCRUD.get_attr_filter(type_id)
if attr_filter:
res = [i for i in res if i.get('attr_name') in attr_filter]
res = [res[i] for i in sorted(res.keys(), reverse=True)]
return total, res
@staticmethod
def get_records_for_relation(start, end, username, page, page_size, operate_type, type_id,
first_ci_id=None, second_ci_id=None):
records = db.session.query(OperationRecord, CIRelationHistory).join(
CIRelationHistory, OperationRecord.id == CIRelationHistory.record_id)
if start:
records = records.filter(OperationRecord.created_at >= start)
if end:
records = records.filter(OperationRecord.created_at <= end)
if type_id:
records = records.filter(OperationRecord.type_id == type_id)
if username:
user = UserCache.get(username)
if user:
records = records.filter(OperationRecord.uid == user.uid)
else:
return abort(404, ErrFormat.user_not_found.format(username))
if operate_type:
records = records.filter(CIRelationHistory.operate_type == operate_type)
if first_ci_id is not None:
records = records.filter(CIRelationHistory.first_ci_id == first_ci_id)
if second_ci_id is not None:
records = records.filter(CIRelationHistory.second_ci_id == second_ci_id)
records = records.order_by(CIRelationHistory.id.desc()).offset(page_size * (page - 1)).limit(page_size).all()
total = len(records)
res = {}
ci_ids = set()
for record in records:
record_id = record.OperationRecord.id
rel_hist = record.CIRelationHistory.to_dict()
ci_ids.add(rel_hist['first_ci_id'])
ci_ids.add(rel_hist['second_ci_id'])
if record_id not in res:
record_dict = record.OperationRecord.to_dict()
record_dict["user"] = UserCache.get(record_dict.get("uid"))
if record_dict["user"]:
record_dict['user'] = record_dict['user'].nickname
res[record_id] = [record_dict, [rel_hist]]
else:
res[record_id][1].append(rel_hist)
res = [res[i] for i in sorted(res.keys(), reverse=True)]
from api.lib.cmdb.ci import CIManager
cis = CIManager().get_cis_by_ids(list(ci_ids),
unique_required=True)
cis = {i['_id']: i for i in cis}
return total, res, cis
@staticmethod
def get_by_ci_id(ci_id):
res = db.session.query(AttributeHistory, Attribute, OperationRecord).join(
Attribute, Attribute.id == AttributeHistory.attr_id).join(
OperationRecord, OperationRecord.id == AttributeHistory.record_id).filter(
AttributeHistory.ci_id == ci_id).order_by(OperationRecord.id.desc())
return [dict(attr_name=i.Attribute.name,
attr_alias=i.Attribute.alias,
operate_type=i.AttributeHistory.operate_type,
username=UserCache.get(i.OperationRecord.uid).nickname,
old=i.AttributeHistory.old,
new=i.AttributeHistory.new,
created_at=i.OperationRecord.created_at.strftime('%Y-%m-%d %H:%M:%S'),
record_id=i.OperationRecord.id,
hid=i.AttributeHistory.id
) for i in res]
AttributeHistory.ci_id == ci_id).order_by(AttributeHistory.id.desc())
from api.lib.cmdb.ci import CIManager
ci = CIManager.get_by_id(ci_id)
attr_filter = CIFilterPermsCRUD.get_attr_filter(ci.type_id) if ci else None
result = []
for i in res:
attr = i.Attribute
if attr_filter and attr.name not in attr_filter:
continue
user = UserCache.get(i.OperationRecord.uid)
hist = i.AttributeHistory
record = i.OperationRecord
item = dict(attr_name=attr.name,
attr_alias=attr.alias,
operate_type=hist.operate_type,
username=user and user.nickname,
old=hist.old,
new=hist.new,
created_at=record.created_at.strftime('%Y-%m-%d %H:%M:%S'),
record_id=record.id,
hid=hist.id
)
result.append(item)
return result
@staticmethod
def get_record_detail(record_id):
from api.lib.cmdb.ci import CIManager
record = OperationRecord.get_by_id(record_id) or abort(404, "Record <{0}> is not found".format(record_id))
record = OperationRecord.get_by_id(record_id) or \
abort(404, ErrFormat.record_not_found.format("id={}".format(record_id)))
username = UserCache.get(record.uid).nickname or UserCache.get(record.uid).username
timestamp = record.created_at.strftime("%Y-%m-%d %H:%M:%S")
@ -101,8 +199,10 @@ class AttributeHistoryManger(object):
return username, timestamp, attr_dict, rel_dict
@staticmethod
def add(ci_id, history_list):
record = OperationRecord.create(uid=g.user.uid)
def add(record_id, ci_id, history_list, type_id=None, flush=False, commit=True):
if record_id is None:
record = OperationRecord.create(uid=g.user.uid, type_id=type_id)
record_id = record.id
for attr_id, operate_type, old, new in history_list or []:
AttributeHistory.create(attr_id=attr_id,
@ -110,7 +210,11 @@ class AttributeHistoryManger(object):
old=json.dumps(old) if isinstance(old, (dict, list)) else old,
new=json.dumps(new) if isinstance(new, (dict, list)) else new,
ci_id=ci_id,
record_id=record.id)
record_id=record_id,
flush=flush,
commit=commit)
return record_id
class CIRelationHistoryManager(object):
@ -124,3 +228,61 @@ class CIRelationHistoryManager(object):
first_ci_id=rel_obj.first_ci_id,
second_ci_id=rel_obj.second_ci_id,
relation_type_id=rel_obj.relation_type_id)
class CITypeHistoryManager(object):
@staticmethod
def get(page, page_size, username=None, type_id=None, operate_type=None):
query = CITypeHistory.get_by(only_query=True)
if type_id is not None:
query = query.filter(CITypeHistory.type_id == type_id)
if username:
user = UserCache.get(username)
if user:
query = query.filter(CITypeHistory.uid == user.uid)
else:
return abort(404, ErrFormat.user_not_found.format(username))
if operate_type is not None:
query = query.filter(CITypeHistory.operate_type == operate_type)
numfound = query.count()
query = query.order_by(CITypeHistory.id.desc())
result = query.offset((page - 1) * page_size).limit(page_size)
result = [i.to_dict() for i in result]
for res in result:
res["user"] = UserCache.get(res.get("uid"))
if res["user"]:
res['user'] = res['user'].nickname
if res.get('attr_id'):
attr = AttributeCache.get(res['attr_id'])
res['attr'] = attr and attr.to_dict()
elif res.get('trigger_id'):
trigger = CITypeTrigger.get_by_id(res['trigger_id'])
res['trigger'] = trigger and trigger.to_dict()
elif res.get('unique_constraint_id'):
unique_constraint = CITypeUniqueConstraint.get_by_id(res['unique_constraint_id'])
res['unique_constraint'] = unique_constraint and unique_constraint.to_dict()
return numfound, result
@staticmethod
def add(operate_type, type_id, attr_id=None, trigger_id=None, unique_constraint_id=None, change=None):
if type_id is None and attr_id is not None:
from api.models.cmdb import CITypeAttribute
type_ids = [i.type_id for i in CITypeAttribute.get_by(attr_id=attr_id, to_dict=False)]
else:
type_ids = [type_id]
for _type_id in type_ids:
payload = dict(operate_type=operate_type,
type_id=_type_id,
uid=g.user.uid,
attr_id=attr_id,
trigger_id=trigger_id,
unique_constraint_id=unique_constraint_id,
change=change)
CITypeHistory.create(**payload)

177
cmdb-api/api/lib/cmdb/perms.py Normal file
View File

@ -0,0 +1,177 @@
# -*- coding:utf-8 -*-
import functools
from flask import abort
from flask import current_app
from flask import g
from flask import request
from api.lib.cmdb.const import ResourceTypeEnum
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.mixin import DBMixin
from api.lib.perm.acl.acl import ACLManager
from api.lib.perm.acl.acl import is_app_admin
from api.lib.perm.acl.acl import validate_permission
from api.models.cmdb import CIFilterPerms
class CIFilterPermsCRUD(DBMixin):
cls = CIFilterPerms
def get(self, type_id):
res = self.cls.get_by(type_id=type_id, to_dict=True)
result = {}
for i in res:
if i['attr_filter']:
i['attr_filter'] = i['attr_filter'].split(',')
if i['rid'] not in result:
result[i['rid']] = i
else:
if i['attr_filter']:
if not result[i['rid']]['attr_filter']:
result[i['rid']]['attr_filter'] = []
result[i['rid']]['attr_filter'] += i['attr_filter']
result[i['rid']]['attr_filter'] = list(set(i['attr_filter']))
if i['ci_filter']:
if not result[i['rid']]['ci_filter']:
result[i['rid']]['ci_filter'] = ""
result[i['rid']]['ci_filter'] += (i['ci_filter'] or "")
return result
def get_by_ids(self, _ids, type_id=None):
if not _ids:
return {}
if type_id is not None:
res = self.cls.get_by(type_id=type_id, __func_in___key_id=_ids, to_dict=True)
else:
res = self.cls.get_by(__func_in___key_id=_ids, to_dict=True)
result = {}
for i in res:
if i['attr_filter']:
i['attr_filter'] = i['attr_filter'].split(',')
if i['type_id'] not in result:
result[i['type_id']] = i
else:
if i['attr_filter']:
if not result[i['type_id']]['attr_filter']:
result[i['type_id']]['attr_filter'] = []
result[i['type_id']]['attr_filter'] += i['attr_filter']
result[i['type_id']]['attr_filter'] = list(set(i['attr_filter']))
if i['ci_filter']:
if not result[i['type_id']]['ci_filter']:
result[i['type_id']]['ci_filter'] = ""
result[i['type_id']]['ci_filter'] += (i['ci_filter'] or "")
return result
@classmethod
def get_attr_filter(cls, type_id):
if is_app_admin('cmdb') or g.user.username in ('worker', 'cmdb_agent'):
return []
res2 = ACLManager('cmdb').get_resources(ResourceTypeEnum.CI_FILTER)
if res2:
type2filter_perms = cls().get_by_ids(list(map(int, [i['name'] for i in res2])), type_id=type_id)
return type2filter_perms.get(type_id, {}).get('attr_filter') or []
def _can_add(self, **kwargs):
ci_filter = kwargs.get('ci_filter')
attr_filter = kwargs.get('attr_filter') or ""
if 'attr_filter' in kwargs:
kwargs['attr_filter'] = kwargs['attr_filter'] or None
if attr_filter:
kwargs['attr_filter'] = ','.join(attr_filter or [])
if ci_filter and not kwargs.get('name'):
return abort(400, ErrFormat.ci_filter_name_cannot_be_empty)
if ci_filter and ci_filter.startswith('q='):
kwargs['ci_filter'] = kwargs['ci_filter'][2:]
return kwargs
def add(self, **kwargs):
kwargs = self._can_add(**kwargs) or kwargs
obj = self.cls.get_by(type_id=kwargs.get('type_id'),
rid=kwargs.get('rid'),
first=True, to_dict=False)
if obj is not None:
obj = obj.update(filter_none=False, **kwargs)
if not obj.attr_filter and not obj.ci_filter:
if current_app.config.get('USE_ACL'):
ACLManager().del_resource(str(obj.id), ResourceTypeEnum.CI_FILTER)
obj.soft_delete()
else:
if not kwargs.get('ci_filter') and not kwargs.get('attr_filter'):
return
obj = self.cls.create(**kwargs)
if current_app.config.get('USE_ACL'):
try:
ACLManager().add_resource(obj.id, ResourceTypeEnum.CI_FILTER)
except:
pass
ACLManager().grant_resource_to_role_by_rid(obj.id,
kwargs.get('rid'),
ResourceTypeEnum.CI_FILTER)
return obj
def _can_update(self, **kwargs):
pass
def _can_delete(self, **kwargs):
pass
def delete(self, **kwargs):
obj = self.cls.get_by(type_id=kwargs.get('type_id'),
rid=kwargs.get('rid'),
first=True, to_dict=False)
if obj is not None:
if current_app.config.get('USE_ACL'):
ACLManager().del_resource(str(obj.id), ResourceTypeEnum.CI_FILTER)
obj.soft_delete()
def has_perm_for_ci(arg_name, resource_type, perm, callback=None, app=None):
def decorator_has_perm(func):
@functools.wraps(func)
def wrapper_has_perm(*args, **kwargs):
if not arg_name:
return
resource = request.view_args.get(arg_name) or request.values.get(arg_name)
if callback is not None and resource:
resource = callback(resource)
if current_app.config.get("USE_ACL") and resource:
if g.user.username == "worker" or g.user.username == "cmdb_agent":
request.values['__is_admin'] = True
return func(*args, **kwargs)
if is_app_admin(app):
request.values['__is_admin'] = True
return func(*args, **kwargs)
validate_permission(resource.name, resource_type, perm, app)
return func(*args, **kwargs)
return wrapper_has_perm
return decorator_has_perm

178
cmdb-api/api/lib/cmdb/preference.py
View File

@ -2,7 +2,6 @@
import copy
import json
import six
import toposort
@ -15,17 +14,25 @@ from api.lib.cmdb.attribute import AttributeManager
from api.lib.cmdb.cache import AttributeCache
from api.lib.cmdb.cache import CITypeAttributesCache
from api.lib.cmdb.cache import CITypeCache
from api.lib.cmdb.const import ResourceTypeEnum, RoleEnum, PermEnum
from api.lib.cmdb.const import PermEnum, ResourceTypeEnum, RoleEnum
from api.lib.cmdb.perms import CIFilterPermsCRUD
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.exception import AbortException
from api.lib.perm.acl.acl import ACLManager
from api.models.cmdb import CITypeAttribute
from api.models.cmdb import CITypeRelation
from api.models.cmdb import PreferenceRelationView
from api.models.cmdb import PreferenceSearchOption
from api.models.cmdb import PreferenceShowAttributes
from api.models.cmdb import PreferenceTreeView
class PreferenceManager(object):
pref_attr_cls = PreferenceShowAttributes
pref_tree_cls = PreferenceTreeView
pref_rel_cls = PreferenceRelationView
pre_so_cls = PreferenceSearchOption
@staticmethod
def get_types(instance=False, tree=False):
types = db.session.query(PreferenceShowAttributes.type_id).filter(
@ -36,19 +43,65 @@ class PreferenceManager(object):
type_ids = list(set([i.type_id for i in types + tree_types]))
return [CITypeCache.get(type_id).to_dict() for type_id in type_ids]
@staticmethod
def get_types2(instance=False, tree=False):
"""
{
self: {instance: [], tree: [], type_id2subs_time: {type_id: subs_time}},
type_id2users: {type_id: []}
}
:param instance:
:param tree:
:return:
"""
result = dict(self=dict(instance=[], tree=[], type_id2subs_time=dict()),
type_id2users=dict())
if instance:
types = db.session.query(PreferenceShowAttributes.type_id,
PreferenceShowAttributes.uid, PreferenceShowAttributes.created_at).filter(
PreferenceShowAttributes.deleted.is_(False)).group_by(
PreferenceShowAttributes.uid, PreferenceShowAttributes.type_id)
for i in types:
if i.uid == g.user.uid:
result['self']['instance'].append(i.type_id)
if str(i.created_at) > str(result['self']['type_id2subs_time'].get(i.type_id, "")):
result['self']['type_id2subs_time'][i.type_id] = i.created_at
result['type_id2users'].setdefault(i.type_id, []).append(i.uid)
if tree:
types = PreferenceTreeView.get_by(to_dict=False)
for i in types:
if i.uid == g.user.uid:
result['self']['tree'].append(i.type_id)
if str(i.created_at) > str(result['self']['type_id2subs_time'].get(i.type_id, "")):
result['self']['type_id2subs_time'][i.type_id] = i.created_at
result['type_id2users'].setdefault(i.type_id, [])
if i.uid not in result['type_id2users'][i.type_id]:
result['type_id2users'][i.type_id].append(i.uid)
return result
@staticmethod
def get_show_attributes(type_id):
if not isinstance(type_id, six.integer_types):
type_id = CITypeCache.get(type_id).id
_type = CITypeCache.get(type_id)
type_id = _type and _type.id
attrs = db.session.query(PreferenceShowAttributes, CITypeAttribute.order).join(
CITypeAttribute, CITypeAttribute.attr_id == PreferenceShowAttributes.attr_id).filter(
PreferenceShowAttributes.uid == g.user.uid).filter(
PreferenceShowAttributes.type_id == type_id).filter(
PreferenceShowAttributes.deleted.is_(False)).filter(CITypeAttribute.deleted.is_(False)).filter(
CITypeAttribute.type_id == type_id).order_by(
CITypeAttribute.order).all()
result = [i.PreferenceShowAttributes.attr.to_dict() for i in attrs]
CITypeAttribute.type_id == type_id).all()
result = []
for i in sorted(attrs, key=lambda x: x.PreferenceShowAttributes.order):
item = i.PreferenceShowAttributes.attr.to_dict()
item.update(dict(is_fixed=i.PreferenceShowAttributes.is_fixed))
result.append(item)
is_subscribed = True
if not attrs:
attrs = db.session.query(CITypeAttribute).filter(
@ -60,15 +113,20 @@ class PreferenceManager(object):
for i in result:
if i["is_choice"]:
i.update(dict(choice_value=AttributeManager.get_choice_values(i["id"], i["value_type"])))
i.update(dict(choice_value=AttributeManager.get_choice_values(
i["id"], i["value_type"], i["choice_web_hook"])))
return is_subscribed, result
@classmethod
def create_or_update_show_attributes(cls, type_id, attr_order):
existed_all = PreferenceShowAttributes.get_by(type_id=type_id, uid=g.user.uid, to_dict=False)
for _attr, order in attr_order:
attr = AttributeCache.get(_attr) or abort(404, "Attribute <{0}> does not exist".format(_attr))
for x, order in attr_order:
if isinstance(x, list):
_attr, is_fixed = x
else:
_attr, is_fixed = x, False
attr = AttributeCache.get(_attr) or abort(404, ErrFormat.attribute_not_found.format("id={}".format(_attr)))
existed = PreferenceShowAttributes.get_by(type_id=type_id,
uid=g.user.uid,
attr_id=attr.id,
@ -78,11 +136,12 @@ class PreferenceManager(object):
PreferenceShowAttributes.create(type_id=type_id,
uid=g.user.uid,
attr_id=attr.id,
order=order)
order=order,
is_fixed=is_fixed)
else:
existed.update(order=order)
existed.update(order=order, is_fixed=is_fixed)
attr_dict = {int(i): j for i, j in attr_order}
attr_dict = {int(i[0]) if isinstance(i, list) else int(i): j for i, j in attr_order}
for i in existed_all:
if i.attr_id not in attr_dict:
i.soft_delete()
@ -92,9 +151,19 @@ class PreferenceManager(object):
res = PreferenceTreeView.get_by(uid=g.user.uid, to_dict=True)
for item in res:
if item["levels"]:
item.update(CITypeCache.get(item['type_id']).to_dict())
item.update(dict(levels=[AttributeCache.get(l).to_dict()
for l in item["levels"].split(",") if AttributeCache.get(l)]))
ci_type = CITypeCache.get(item['type_id']).to_dict()
attr_filter = CIFilterPermsCRUD.get_attr_filter(ci_type['id'])
ci_type.pop('id', None)
ci_type.pop('created_at', None)
ci_type.pop('updated_at', None)
item.update(ci_type)
_levels = []
for i in item["levels"]:
attr = AttributeCache.get(i)
if attr and (not attr_filter or attr.name in attr_filter):
_levels.append(attr.to_dict())
item.update(dict(levels=_levels))
return res
@ -105,8 +174,7 @@ class PreferenceManager(object):
for attr in attrs:
attr = AttributeCache.get(attr.attr_id)
if i == attr.id or i == attr.name or i == attr.alias:
levels[idx] = str(attr.id)
levels = ",".join(levels)
levels[idx] = attr.id
existed = PreferenceTreeView.get_by(uid=g.user.uid, type_id=type_id, to_dict=False, first=True)
if existed is not None:
@ -124,9 +192,9 @@ class PreferenceManager(object):
if current_app.config.get("USE_ACL"):
for i in _views:
try:
if ACLManager().has_permission(i.get('name'),
ResourceTypeEnum.RELATION_VIEW,
PermEnum.READ):
if i.get('is_public') or ACLManager().has_permission(i.get('name'),
ResourceTypeEnum.RELATION_VIEW,
PermEnum.READ):
views.append(i)
except AbortException:
pass
@ -137,7 +205,7 @@ class PreferenceManager(object):
result = dict()
name2id = list()
for view in views:
view2cr_ids.setdefault(view['name'], []).extend(json.loads(view['cr_ids']))
view2cr_ids.setdefault(view['name'], []).extend(view['cr_ids'])
name2id.append([view['name'], view['id']])
id2type = dict()
@ -179,14 +247,14 @@ class PreferenceManager(object):
return result, id2type, sorted(name2id, key=lambda x: x[1])
@classmethod
def create_or_update_relation_view(cls, name, cr_ids):
def create_or_update_relation_view(cls, name, cr_ids, is_public=False):
if not cr_ids:
return abort(400, "Node must be selected")
return abort(400, ErrFormat.preference_relation_view_node_required)
existed = PreferenceRelationView.get_by(name=name, to_dict=False, first=True)
current_app.logger.debug(existed)
if existed is None:
PreferenceRelationView.create(name=name, cr_ids=json.dumps(cr_ids))
PreferenceRelationView.create(name=name, cr_ids=cr_ids, uid=g.user.uid, is_public=is_public)
if current_app.config.get("USE_ACL"):
ACLManager().add_resource(name, ResourceTypeEnum.RELATION_VIEW)
@ -206,3 +274,65 @@ class PreferenceManager(object):
ACLManager().del_resource(name, ResourceTypeEnum.RELATION_VIEW)
return name
@staticmethod
def get_search_option(**kwargs):
query = PreferenceSearchOption.get_by(only_query=True)
query = query.filter(PreferenceSearchOption.uid == g.user.uid)
for k in kwargs:
if hasattr(PreferenceSearchOption, k) and kwargs[k]:
query = query.filter(getattr(PreferenceSearchOption, k) == kwargs[k])
return [i.to_dict() for i in query]
@staticmethod
def add_search_option(**kwargs):
kwargs['uid'] = g.user.uid
existed = PreferenceSearchOption.get_by(uid=g.user.uid,
name=kwargs.get('name'),
prv_id=kwargs.get('prv_id'),
ptv_id=kwargs.get('ptv_id'),
type_id=kwargs.get('type_id'),
)
if existed:
return abort(400, ErrFormat.preference_search_option_exists)
return PreferenceSearchOption.create(**kwargs)
@staticmethod
def update_search_option(_id, **kwargs):
existed = PreferenceSearchOption.get_by_id(_id) or abort(404, ErrFormat.preference_search_option_not_found)
if g.user.uid != existed.uid:
return abort(400, ErrFormat.no_permission2)
other = PreferenceSearchOption.get_by(uid=g.user.uid,
name=kwargs.get('name'),
prv_id=kwargs.get('prv_id'),
ptv_id=kwargs.get('ptv_id'),
type_id=kwargs.get('type_id'),
)
if other.id != _id:
return abort(400, ErrFormat.preference_search_option_exists)
return existed.update(**kwargs)
@staticmethod
def delete_search_option(_id):
existed = PreferenceSearchOption.get_by_id(_id) or abort(404, ErrFormat.preference_search_option_not_found)
if g.user.uid != existed.uid:
return abort(400, ErrFormat.no_permission2)
existed.soft_delete()
@staticmethod
def delete_by_type_id(type_id, uid):
for i in PreferenceShowAttributes.get_by(type_id=type_id, uid=uid, to_dict=False):
i.soft_delete()
for i in PreferenceTreeView.get_by(type_id=type_id, uid=uid, to_dict=False):
i.soft_delete()

63
cmdb-api/api/lib/cmdb/query_sql.py Normal file
View File

@ -0,0 +1,63 @@
# -*- coding:utf-8 -*-
QUERY_CIS_BY_VALUE_TABLE = """
SELECT attr.name AS attr_name,
attr.alias AS attr_alias,
attr.value_type,
attr.is_list,
c_cis.type_id,
{0}.ci_id,
{0}.attr_id,
{0}.value
FROM {0}
INNER JOIN c_cis ON {0}.ci_id=c_cis.id
AND {0}.`ci_id` IN ({1})
INNER JOIN c_attributes as attr ON attr.id = {0}.attr_id
"""
# {2}: value_table
QUERY_CIS_BY_IDS = """
SELECT A.ci_id,
A.type_id,
A.attr_id,
A.attr_name,
A.attr_alias,
A.value,
A.value_type,
A.is_list
FROM
({1}) AS A {0}
ORDER BY A.ci_id;
"""
FACET_QUERY1 = """
SELECT {0}.value,
count({0}.ci_id)
FROM {0}
INNER JOIN c_attributes AS attr ON attr.id={0}.attr_id
WHERE attr.name="{1}"
GROUP BY {0}.ci_id;
"""
FACET_QUERY = """
SELECT {0}.value,
count({0}.ci_id)
FROM {0}
INNER JOIN ({1}) AS F ON F.ci_id={0}.ci_id
WHERE {0}.attr_id={2:d}
GROUP BY {0}.value
"""
QUERY_CI_BY_ATTR_NAME = """
SELECT {0}.ci_id
FROM {0}
WHERE {0}.attr_id={1:d}
AND {0}.value {2}
"""
QUERY_CI_BY_TYPE = """
SELECT c_cis.id AS ci_id
FROM c_cis
WHERE c_cis.type_id in ({0})
"""

13
cmdb-api/api/lib/cmdb/relation_type.py
View File

@ -3,10 +3,13 @@
from flask import abort
from api.lib.cmdb.resp_format import ErrFormat
from api.models.cmdb import RelationType
class RelationTypeManager(object):
cls = RelationType
@staticmethod
def get_all():
return RelationType.get_by(to_dict=False)
@ -21,17 +24,21 @@ class RelationTypeManager(object):
@staticmethod
def add(name):
RelationType.get_by(name=name, first=True, to_dict=False) and abort(400, "It's already existed")
RelationType.get_by(name=name, first=True, to_dict=False) and \
abort(400, ErrFormat.relation_type_exists.format(name))
return RelationType.create(name=name)
@staticmethod
def update(rel_id, name):
existed = RelationType.get_by_id(rel_id) or abort(404, "RelationType <{0}> does not exist".format(rel_id))
existed = RelationType.get_by_id(rel_id) or \
abort(404, ErrFormat.relation_type_not_found.format("id={}".format(rel_id)))
return existed.update(name=name)
@staticmethod
def delete(rel_id):
existed = RelationType.get_by_id(rel_id) or abort(404, "RelationType <{0}> does not exist".format(rel_id))
existed = RelationType.get_by_id(rel_id) or \
abort(404, ErrFormat.relation_type_not_found.format("id={}".format(rel_id)))
existed.soft_delete()

93
cmdb-api/api/lib/cmdb/resp_format.py Normal file
View File

@ -0,0 +1,93 @@
# -*- coding:utf-8 -*-
from api.lib.resp_format import CommonErrFormat
class ErrFormat(CommonErrFormat):
invalid_relation_type = "无效的关系类型: {}"
ci_type_not_found = "模型不存在!"
argument_attributes_must_be_list = "参数 attributes 类型必须是列表"
argument_file_not_found = "文件似乎并未上传"
attribute_not_found = "属性 {} 不存在!"
attribute_value_type_cannot_change = "属性的值类型不允许修改!"
attribute_list_value_cannot_change = "多值不被允许修改!"
attribute_index_cannot_change = "修改索引 非管理员不被允许!"
attribute_index_change_failed = "索引切换失败!"
invalid_choice_values = "预定义值的类型不对！"
attribute_name_duplicate = "重复的属性名 {}"
add_attribute_failed = "创建属性 {} 失败!"
update_attribute_failed = "修改属性 {} 失败!"
cannot_edit_attribute = "您没有权限修改该属性!"
cannot_delete_attribute = "您没有权限删除该属性!"
attribute_name_cannot_be_builtin = "属性字段名不能是内置字段: id, _id, ci_id, type, _type, ci_type"
ci_not_found = "CI {} 不存在"
unique_constraint = "多属性联合唯一校验不通过: {}"
unique_value_not_found = "模型的主键 {} 不存在!"
unique_key_required = "主键字段 {} 缺失"
ci_is_already_existed = "CI 已经存在!"
relation_constraint = "关系约束: {}, 校验失败 "
relation_not_found = "CI关系: {} 不存在"
ci_search_Parentheses_invalid = "搜索表达式里小括号前不支持: 或、非"
ci_type_not_found2 = "模型 {} 不存在"
ci_type_is_already_existed = "模型 {} 已经存在"
unique_key_not_define = "主键未定义或者已被删除"
only_owner_can_delete = "只有创建人才能删除它!"
ci_exists_and_cannot_delete_type = "因为CI已经存在，不能删除模型"
ci_type_group_not_found = "模型分组 {} 不存在"
ci_type_group_exists = "模型分组 {} 已经存在"
ci_type_relation_not_found = "模型关系 {} 不存在"
ci_type_attribute_group_duplicate = "属性分组 {} 已存在"
ci_type_attribute_group_not_found = "属性分组 {} 不存在"
ci_type_group_attribute_not_found = "属性组<{0}> - 属性<{1}> 不存在"
unique_constraint_duplicate = "唯一约束已经存在!"
unique_constraint_invalid = "唯一约束的属性不能是 JSON 和 多值"
ci_type_trigger_duplicate = "重复的触发器"
ci_type_trigger_not_found = "触发器 {} 不存在"
record_not_found = "操作记录 {} 不存在"
cannot_delete_unique = "不能删除唯一标识"
cannot_delete_default_order_attr = "不能删除默认排序的属性"
preference_relation_view_node_required = "没有选择节点"
preference_search_option_not_found = "该搜索选项不存在!"
preference_search_option_exists = "该搜索选项命名重复!"
relation_type_exists = "关系类型 {} 已经存在"
relation_type_not_found = "关系类型 {} 不存在"
attribute_value_invalid = "无效的属性值: {}"
attribute_value_invalid2 = "{} 无效的值: {}"
not_in_choice_values = "{} 不在预定义值里"
attribute_value_unique_required = "属性 {} 的值必须是唯一的, 当前值 {} 已存在"
attribute_value_required = "属性 {} 值必须存在"
attribute_value_unknown_error = "新增或者修改属性值未知错误: {}"
custom_name_duplicate = "订制名重复"
limit_ci_type = "模型数超过限制: {}"
limit_ci = "CI数超过限制: {}"
adr_duplicate = "自动发现规则: {} 已经存在!"
adr_not_found = "自动发现规则: {} 不存在!"
adr_referenced = "该自动发现规则被模型引用, 不能删除!"
ad_duplicate = "自动发现规则的应用不能重复定义!"
ad_not_found = "您要修改的自动发现: {} 不存在!"
ad_not_unique_key = "属性字段没有包括唯一标识: {}"
adc_not_found = "自动发现的实例不存在!"
adt_not_found = "模型并未关联该自动发现!"
adt_secret_no_permission = "只有创建人才能修改Secret!"
cannot_delete_adt = "该规则已经有自动发现的实例, 不能被删除!"
adr_default_ref_once = "该默认的自动发现规则 已经被模型 {} 引用!"
adr_unique_key_required = "unique_key方法必须返回非空字符串!"
adr_plugin_attributes_list_required = "attributes方法必须返回的是list"
adr_plugin_attributes_list_no_empty = "attributes方法返回的list不能为空!"
adt_target_all_no_permission = "只有管理员才可以定义执行机器为: 所有节点!"
adt_target_expr_no_permission = "执行机器权限检查不通过: {}"
ci_filter_name_cannot_be_empty = "CI过滤授权 必须命名!"
ci_filter_perm_cannot_or_query = "CI过滤授权 暂时不支持 或 查询"
ci_filter_perm_attr_no_permission = "您没有属性 {} 的操作权限!"
ci_filter_perm_ci_no_permission = "您没有该CI的操作权限!"

24
cmdb-api/api/lib/cmdb/search/ci/__init__.py
View File

@ -1,3 +1,25 @@
# -*- coding:utf-8 -*-
__all__ = ['db', 'es']
__all__ = ['db', 'es', 'search']
from flask import current_app
from api.lib.cmdb.const import RetKey
from api.lib.cmdb.search.ci.db.search import Search as SearchFromDB
from api.lib.cmdb.search.ci.es.search import Search as SearchFromES
def search(query=None,
fl=None,
facet=None,
page=1,
ret_key=RetKey.NAME,
count=1,
sort=None,
excludes=None):
if current_app.config.get("USE_ES"):
s = SearchFromES(query, fl, facet, page, ret_key, count, sort)
else:
s = SearchFromDB(query, fl, facet, page, ret_key, count, sort, excludes=excludes)
return s

43
cmdb-api/api/lib/cmdb/search/ci/db/query_sql.py
View File

@ -2,7 +2,6 @@
from __future__ import unicode_literals
QUERY_CIS_BY_VALUE_TABLE = """
SELECT attr.name AS attr_name,
attr.alias AS attr_alias,
@ -58,9 +57,51 @@ QUERY_CI_BY_ATTR_NAME = """
AND {0}.value {2}
"""
QUERY_CI_BY_ID = """
SELECT c_cis.id as ci_id
FROM c_cis
WHERE c_cis.id={}
"""
QUERY_CI_BY_TYPE = """
SELECT c_cis.id AS ci_id
FROM c_cis
WHERE c_cis.type_id in ({0})
"""
QUERY_UNION_CI_ATTRIBUTE_IS_NULL = """
SELECT *
FROM (
SELECT c_cis.id AS ci_id
FROM c_cis
WHERE c_cis.type_id IN ({0})
) {3}
LEFT JOIN (
SELECT {1}.ci_id
FROM {1}
WHERE {1}.attr_id = {2}
AND {1}.value LIKE "%"
) {4} USING (ci_id)
WHERE {4}.ci_id IS NULL
"""
QUERY_CI_BY_NO_ATTR = """
SELECT *
FROM
(SELECT c_value_index_texts.ci_id
FROM c_value_index_texts
WHERE c_value_index_texts.value LIKE "{0}"
UNION
SELECT c_value_index_integers.ci_id
FROM c_value_index_integers
WHERE c_value_index_integers.value LIKE "{0}"
UNION
SELECT c_value_index_floats.ci_id
FROM c_value_index_floats
WHERE c_value_index_floats.value LIKE "{0}"
UNION
SELECT c_value_index_datetime.ci_id
FROM c_value_index_datetime
WHERE c_value_index_datetime.value LIKE "{0}") AS {1}
GROUP BY {1}.ci_id
"""

311
cmdb-api/api/lib/cmdb/search/ci/db/search.py
View File

@ -3,23 +3,33 @@
from __future__ import unicode_literals
import copy
import time
from flask import current_app
from flask import g
from jinja2 import Template
from api.extensions import db
from api.lib.cmdb.cache import AttributeCache
from api.lib.cmdb.cache import CITypeCache
from api.lib.cmdb.ci import CIManager
from api.lib.cmdb.const import PermEnum
from api.lib.cmdb.const import ResourceTypeEnum
from api.lib.cmdb.const import RetKey
from api.lib.cmdb.const import ValueTypeEnum
from api.lib.cmdb.perms import CIFilterPermsCRUD
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.cmdb.search import SearchError
from api.lib.cmdb.search.ci.db.query_sql import FACET_QUERY
from api.lib.cmdb.search.ci.db.query_sql import QUERY_CI_BY_ATTR_NAME
from api.lib.cmdb.search.ci.db.query_sql import QUERY_CI_BY_ID
from api.lib.cmdb.search.ci.db.query_sql import QUERY_CI_BY_NO_ATTR
from api.lib.cmdb.search.ci.db.query_sql import QUERY_CI_BY_TYPE
from api.lib.cmdb.search.ci.db.query_sql import QUERY_UNION_CI_ATTRIBUTE_IS_NULL
from api.lib.cmdb.utils import TableMap
from api.lib.perm.acl.acl import ACLManager
from api.lib.perm.acl.acl import is_app_admin
from api.lib.utils import handle_arg_list
from api.models.cmdb import CI
class Search(object):
@ -30,9 +40,11 @@ class Search(object):
ret_key=RetKey.NAME,
count=1,
sort=None,
ci_ids=None):
ci_ids=None,
excludes=None):
self.orig_query = query
self.fl = fl
self.fl = fl or []
self.excludes = excludes or []
self.facet_field = facet_field
self.page = page
self.ret_key = ret_key
@ -43,11 +55,17 @@ class Search(object):
self.type_id_list = []
self.only_type_query = False
self.valid_type_names = []
self.type2filter_perms = dict()
@staticmethod
def _operator_proc(key):
operator = "&"
if key.startswith("+"):
key = key[1:].strip()
elif key.startswith("-~"):
operator = "|~"
key = key[2:].strip()
elif key.startswith("-"):
operator = "|"
key = key[1:].strip()
@ -70,14 +88,41 @@ class Search(object):
if attr:
return attr.name, attr.value_type, operator, attr
else:
raise SearchError("{0} is not existed".format(key))
raise SearchError(ErrFormat.attribute_not_found.format(key))
def _type_query_handler(self, v):
def _type_query_handler(self, v, queries):
new_v = v[1:-1].split(";") if v.startswith("(") and v.endswith(")") else [v]
for _v in new_v:
ci_type = CITypeCache.get(_v)
if len(new_v) == 1 and not self.sort and ci_type.default_order_attr:
self.sort = ci_type.default_order_attr
if ci_type is not None:
self.type_id_list.append(str(ci_type.id))
if self.valid_type_names == "ALL" or ci_type.name in self.valid_type_names:
self.type_id_list.append(str(ci_type.id))
if ci_type.id in self.type2filter_perms:
ci_filter = self.type2filter_perms[ci_type.id].get('ci_filter')
if ci_filter:
sub = []
ci_filter = Template(ci_filter).render(user=g.user)
for i in ci_filter.split(','):
if i.startswith("~") and not sub:
queries.append(i)
else:
sub.append(i)
if sub:
queries.append(dict(operator="&", queries=sub))
if self.type2filter_perms[ci_type.id].get('attr_filter'):
if not self.fl:
self.fl = set(self.type2filter_perms[ci_type.id]['attr_filter'])
else:
self.fl = set(self.fl) & set(self.type2filter_perms[ci_type.id]['attr_filter'])
else:
raise SearchError(ErrFormat.no_permission.format(ci_type.alias, PermEnum.READ))
else:
raise SearchError(ErrFormat.ci_type_not_found2.format(_v))
if self.type_id_list:
type_ids = ",".join(self.type_id_list)
@ -89,24 +134,32 @@ class Search(object):
return ""
@staticmethod
def _in_query_handler(attr, v):
def _id_query_handler(v):
return QUERY_CI_BY_ID.format(v)
@staticmethod
def _in_query_handler(attr, v, is_not):
new_v = v[1:-1].split(";")
table_name = TableMap(attr_name=attr.name).table_name
in_query = " OR {0}.value ".format(table_name).join(['LIKE "{0}"'.format(_v.replace("*", "%")) for _v in new_v])
table_name = TableMap(attr=attr).table_name
in_query = " OR {0}.value ".format(table_name).join(['{0} "{1}"'.format(
"NOT LIKE" if is_not else "LIKE",
_v.replace("*", "%")) for _v in new_v])
_query_sql = QUERY_CI_BY_ATTR_NAME.format(table_name, attr.id, in_query)
return _query_sql
@staticmethod
def _range_query_handler(attr, v):
def _range_query_handler(attr, v, is_not):
start, end = [x.strip() for x in v[1:-1].split("_TO_")]
table_name = TableMap(attr_name=attr.name).table_name
range_query = "BETWEEN '{0}' AND '{1}'".format(start.replace("*", "%"), end.replace("*", "%"))
table_name = TableMap(attr=attr).table_name
range_query = "{0} '{1}' AND '{2}'".format(
"NOT BETWEEN" if is_not else "BETWEEN",
start.replace("*", "%"), end.replace("*", "%"))
_query_sql = QUERY_CI_BY_ATTR_NAME.format(table_name, attr.id, range_query)
return _query_sql
@staticmethod
def _comparison_query_handler(attr, v):
table_name = TableMap(attr_name=attr.name).table_name
table_name = TableMap(attr=attr).table_name
if v.startswith(">=") or v.startswith("<="):
comparison_query = "{0} '{1}'".format(v[:2], v[2:].replace("*", "%"))
else:
@ -154,18 +207,44 @@ class Search(object):
"INNER JOIN c_cis on c_cis.id=B.ci_id "
"ORDER BY B.ci_id {1} LIMIT {0:d}, {2};".format((self.page - 1) * self.count, sort_type, self.count))
def __sort_by_type(self, sort_type, query_sql):
ret_sql = "SELECT SQL_CALC_FOUND_ROWS DISTINCT B.ci_id FROM ({0}) AS B {1}"
if self.type_id_list:
self.query_sql = "SELECT B.ci_id FROM ({0}) AS B {1}".format(
query_sql,
"INNER JOIN c_cis on c_cis.id=B.ci_id WHERE c_cis.type_id IN ({0}) ".format(
",".join(self.type_id_list)))
return ret_sql.format(
query_sql,
"INNER JOIN c_cis on c_cis.id=B.ci_id WHERE c_cis.type_id IN ({3}) "
"ORDER BY c_cis.type_id {1} LIMIT {0:d}, {2};".format(
(self.page - 1) * self.count, sort_type, self.count, ",".join(self.type_id_list)))
else:
self.query_sql = "SELECT B.ci_id FROM ({0}) AS B {1}".format(
query_sql,
"INNER JOIN c_cis on c_cis.id=B.ci_id ")
return ret_sql.format(
query_sql,
"INNER JOIN c_cis on c_cis.id=B.ci_id "
"ORDER BY c_cis.type_id {1} LIMIT {0:d}, {2};".format(
(self.page - 1) * self.count, sort_type, self.count))
def __sort_by_field(self, field, sort_type, query_sql):
attr = AttributeCache.get(field)
attr_id = attr.id
table_name = TableMap(attr_name=attr.name).table_name
table_name = TableMap(attr=attr).table_name
_v_query_sql = """SELECT {0}.ci_id, {1}.value
FROM ({2}) AS {0} INNER JOIN {1} ON {1}.ci_id = {0}.ci_id
WHERE {1}.attr_id = {3}""".format("ALIAS", table_name, query_sql, attr_id)
new_table = _v_query_sql
if self.only_type_query or not self.type_id_list:
return "SELECT SQL_CALC_FOUND_ROWS DISTINCT C.ci_id, C.value " \
return "SELECT SQL_CALC_FOUND_ROWS DISTINCT C.ci_id " \
"FROM ({0}) AS C " \
"ORDER BY C.value {2} " \
"LIMIT {1:d}, {3};".format(new_table, (self.page - 1) * self.count, sort_type, self.count)
@ -176,7 +255,7 @@ class Search(object):
INNER JOIN c_cis on c_cis.id=C.ci_id
WHERE c_cis.type_id IN ({1})""".format(new_table, ",".join(self.type_id_list))
return """SELECT SQL_CALC_FOUND_ROWS DISTINCT C.ci_id, C.value
return """SELECT SQL_CALC_FOUND_ROWS DISTINCT C.ci_id
FROM ({0}) AS C
INNER JOIN c_cis on c_cis.id=C.ci_id
WHERE c_cis.type_id IN ({4})
@ -192,6 +271,8 @@ class Search(object):
if field in ("_id", "ci_id") or not field:
return self.__sort_by_id(sort_type, query_sql)
elif field in ("_type", "ci_type"):
return self.__sort_by_type(sort_type, query_sql)
else:
return self.__sort_by_field(field, sort_type, query_sql)
@ -201,7 +282,7 @@ class Search(object):
query_sql = """SELECT * FROM ({0}) as {1}
INNER JOIN ({2}) as {3} USING(ci_id)""".format(query_sql, alias, _query_sql, alias + "A")
elif operator == "|":
elif operator == "|" or operator == "|~":
query_sql = "SELECT * FROM ({0}) as {1} UNION ALL ({2})".format(query_sql, alias, _query_sql)
elif operator == "~":
@ -225,54 +306,142 @@ class Search(object):
return numfound, res
def __get_types_has_read(self):
"""
:return: _type:(type1;type2)
"""
acl = ACLManager('cmdb')
res = acl.get_resources(ResourceTypeEnum.CI)
self.valid_type_names = {i['name'] for i in res if PermEnum.READ in i['permissions']}
res2 = acl.get_resources(ResourceTypeEnum.CI_FILTER)
if res2:
self.type2filter_perms = CIFilterPermsCRUD().get_by_ids(list(map(int, [i['name'] for i in res2])))
return "_type:({})".format(";".join(self.valid_type_names))
def __confirm_type_first(self, queries):
has_type = False
result = []
sub = {}
id_query = None
for q in queries:
if q.startswith("_type"):
queries.remove(q)
queries.insert(0, q)
has_type = True
result.insert(0, q)
if len(queries) == 1 or queries[1].startswith("-") or queries[1].startswith("~"):
self.only_type_query = True
return queries
elif q.startswith("_id") and len(q.split(':')) == 2:
id_query = int(q.split(":")[1]) if q.split(":")[1].isdigit() else None
result.append(q)
elif q.startswith("(") or q[1:].startswith("(") or q[2:].startswith("("):
if not q.startswith("("):
raise SearchError(ErrFormat.ci_search_Parentheses_invalid)
def __query_build_by_field(self, queries):
query_sql, alias, operator = "", "A", "&"
is_first, only_type_query_special = True, True
operator, q = self._operator_proc(q)
if q.endswith(")"):
result.append(dict(operator=operator, queries=[q[1:-1]]))
sub = dict(operator=operator, queries=[q[1:]])
elif q.endswith(")") and sub:
sub['queries'].append(q[:-1])
result.append(copy.deepcopy(sub))
sub = {}
elif sub:
sub['queries'].append(q)
else:
result.append(q)
_is_app_admin = is_app_admin('cmdb') or g.user.username == "worker"
if result and not has_type and not _is_app_admin:
type_q = self.__get_types_has_read()
if id_query:
ci = CIManager.get_by_id(id_query)
if not ci:
raise SearchError(ErrFormat.ci_not_found.format(id_query))
result.insert(0, "_type:{}".format(ci.type_id))
else:
result.insert(0, type_q)
elif _is_app_admin:
self.valid_type_names = "ALL"
else:
self.__get_types_has_read()
current_app.logger.warning(result)
return result
def __query_by_attr(self, q, queries, alias):
k = q.split(":")[0].strip()
v = "\:".join(q.split(":")[1:]).strip()
v = v.replace("'", "\\'")
v = v.replace('"', '\\"')
field, field_type, operator, attr = self._attr_name_proc(k)
if field == "_type":
_query_sql = self._type_query_handler(v, queries)
elif field == "_id":
_query_sql = self._id_query_handler(v)
elif field:
if attr is None:
raise SearchError(ErrFormat.attribute_not_found.format(field))
is_not = True if operator == "|~" else False
# in query
if v.startswith("(") and v.endswith(")"):
_query_sql = self._in_query_handler(attr, v, is_not)
# range query
elif v.startswith("[") and v.endswith("]") and "_TO_" in v:
_query_sql = self._range_query_handler(attr, v, is_not)
# comparison query
elif v.startswith(">=") or v.startswith("<=") or v.startswith(">") or v.startswith("<"):
_query_sql = self._comparison_query_handler(attr, v)
else:
table_name = TableMap(attr=attr).table_name
if is_not and v == "*" and self.type_id_list: # special handle
_query_sql = QUERY_UNION_CI_ATTRIBUTE_IS_NULL.format(
",".join(self.type_id_list),
table_name,
attr.id,
alias,
alias + 'A'
)
alias += "AA"
else:
_query_sql = QUERY_CI_BY_ATTR_NAME.format(
table_name,
attr.id,
'{0} "{1}"'.format("NOT LIKE" if is_not else "LIKE", v.replace("*", "%")))
else:
raise SearchError(ErrFormat.argument_invalid.format("q"))
return alias, _query_sql, operator
def __query_build_by_field(self, queries, is_first=True, only_type_query_special=True, alias='A', operator='&'):
query_sql = ""
for q in queries:
_query_sql = ""
if ":" in q:
k = q.split(":")[0].strip()
v = "\:".join(q.split(":")[1:]).strip()
current_app.logger.debug(v)
field, field_type, operator, attr = self._attr_name_proc(k)
if field == "_type":
_query_sql = self._type_query_handler(v)
current_app.logger.debug(_query_sql)
elif field == "_id": # exclude all others
ci = CI.get_by_id(v)
if ci is not None:
return 1, [str(v)]
elif field:
if attr is None:
raise SearchError("{0} is not found".format(field))
if isinstance(q, dict):
alias, _query_sql, operator = self.__query_build_by_field(q['queries'], True, True, alias)
current_app.logger.info(_query_sql)
current_app.logger.info((operator, is_first, alias))
operator = q['operator']
# in query
if v.startswith("(") and v.endswith(")"):
_query_sql = self._in_query_handler(attr, v)
# range query
elif v.startswith("[") and v.endswith("]") and "_TO_" in v:
_query_sql = self._range_query_handler(attr, v)
# comparison query
elif v.startswith(">=") or v.startswith("<=") or v.startswith(">") or v.startswith("<"):
_query_sql = self._comparison_query_handler(attr, v)
else:
table_name = TableMap(attr_name=attr.name).table_name
_query_sql = QUERY_CI_BY_ATTR_NAME.format(
table_name, attr.id, 'LIKE "{0}"'.format(v.replace("*", "%")))
else:
raise SearchError("argument q format invalid: {0}".format(q))
elif ":" in q and not q.startswith("*"):
alias, _query_sql, operator = self.__query_by_attr(q, queries, alias)
elif q == "*":
continue
elif q:
raise SearchError("argument q format invalid: {0}".format(q))
q = q.replace("'", "\\'")
q = q.replace('"', '\\"')
q = q.replace("*", "%").replace('\\n', '%')
_query_sql = QUERY_CI_BY_NO_ATTR.format(q, alias)
if is_first and _query_sql and not self.only_type_query:
query_sql = "SELECT * FROM ({0}) AS {1}".format(_query_sql, alias)
@ -285,7 +454,8 @@ class Search(object):
elif _query_sql:
query_sql = self._wrap_sql(operator, alias, _query_sql, query_sql)
alias += "AA"
return None, query_sql
return alias, query_sql, operator
def _filter_ids(self, query_sql):
if self.ci_ids:
@ -294,21 +464,36 @@ class Search(object):
return query_sql
@staticmethod
def _extra_handle_query_expr(args): # \, or ,
result = []
if args:
result.append(args[0])
for arg in args[1:]:
if result[-1].endswith('\\'):
result[-1] = ",".join([result[-1].rstrip('\\'), arg])
# elif ":" not in arg:
# result[-1] = ",".join([result[-1], arg])
else:
result.append(arg)
return result
def _query_build_raw(self):
queries = handle_arg_list(self.orig_query)
queries = self._extra_handle_query_expr(queries)
queries = self.__confirm_type_first(queries)
current_app.logger.debug(queries)
ret, query_sql = self.__query_build_by_field(queries)
if ret is not None:
return ret, query_sql
_, query_sql, _ = self.__query_build_by_field(queries)
s = time.time()
if query_sql:
query_sql = self._filter_ids(query_sql)
self.query_sql = query_sql
current_app.logger.debug(query_sql)
# current_app.logger.debug(query_sql)
numfound, res = self._execute_sql(query_sql)
current_app.logger.debug("query ci ids is: {0}".format(time.time() - s))
return numfound, [_res[0] for _res in res]
@ -320,9 +505,9 @@ class Search(object):
for f in self.facet_field:
k, field_type, _, attr = self._attr_name_proc(f)
if k:
table_name = TableMap(attr_name=k).table_name
table_name = TableMap(attr=attr).table_name
query_sql = FACET_QUERY.format(table_name, self.query_sql, attr.id)
current_app.logger.debug(query_sql)
# current_app.logger.debug(query_sql)
result = db.session.execute(query_sql).fetchall()
facet[k] = result
@ -356,7 +541,7 @@ class Search(object):
response, counter = [], {}
if ci_ids:
response = CIManager.get_cis_by_ids(ci_ids, ret_key=self.ret_key, fields=_fl)
response = CIManager.get_cis_by_ids(ci_ids, ret_key=self.ret_key, fields=_fl, excludes=self.excludes)
for res in response:
ci_type = res.get("ci_type")
if ci_type not in counter.keys():

147
cmdb-api/api/lib/cmdb/search/ci/es/search.py
View File

@ -10,6 +10,7 @@ from api.extensions import es
from api.lib.cmdb.cache import AttributeCache
from api.lib.cmdb.const import RetKey
from api.lib.cmdb.const import ValueTypeEnum
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.cmdb.search import SearchError
from api.lib.utils import handle_arg_list
@ -22,9 +23,11 @@ class Search(object):
ret_key=RetKey.NAME,
count=1,
sort=None,
ci_ids=None):
ci_ids=None,
excludes=None):
self.orig_query = query
self.fl = fl
self.fl = fl or []
self.excludes = excludes or []
self.facet_field = facet_field
self.page = page
self.ret_key = ret_key
@ -39,6 +42,9 @@ class Search(object):
operator = "&"
if key.startswith("+"):
key = key[1:].strip()
elif key.startswith("-~"):
operator = "|~"
key = key[2:].strip()
elif key.startswith("-"):
operator = "|"
key = key[1:].strip()
@ -53,6 +59,8 @@ class Search(object):
return self.query['query']['bool']['must']
elif operator == "|":
return self.query['query']['bool']['should']
elif operator == "|~":
return self.query['query']['bool']['should']
else:
return self.query['query']['bool']['must_not']
@ -69,9 +77,9 @@ class Search(object):
if attr:
return attr.name, attr.value_type, operator
else:
raise SearchError("{0} is not existed".format(key))
raise SearchError(ErrFormat.attribute_not_found.format(key))
def _in_query_handle(self, attr, v):
def _in_query_handle(self, attr, v, is_not):
terms = v[1:-1].split(";")
operator = "|"
if attr in ('_type', 'ci_type', 'type_id') and terms and terms[0].isdigit():
@ -79,11 +87,27 @@ class Search(object):
terms = map(int, terms)
current_app.logger.warning(terms)
for term in terms:
self._operator2query(operator).append({
"term": {
attr: term
}
})
if is_not:
self._operator2query(operator).append({
"bool": {
"must_not": [
{
"term": {
attr: term
}
}
]
}
})
else:
self._operator2query(operator).append({
"term": {
attr: term
}
})
def _filter_ids(self):
if self.ci_ids:
@ -95,18 +119,36 @@ class Search(object):
return int(float(s))
return s
def _range_query_handle(self, attr, v, operator):
def _range_query_handle(self, attr, v, operator, is_not):
left, right = v.split("_TO_")
left, right = left.strip()[1:], right.strip()[:-1]
self._operator2query(operator).append({
"range": {
attr: {
"lte": self._digit(right),
"gte": self._digit(left),
"boost": 2.0
if is_not:
self._operator2query(operator).append({
"bool": {
"must_not": [
{
"range": {
attr: {
"lte": self._digit(right),
"gte": self._digit(left),
"boost": 2.0
}
}
}
]
}
}
})
})
else:
self._operator2query(operator).append({
"range": {
attr: {
"lte": self._digit(right),
"gte": self._digit(left),
"boost": 2.0
}
}
})
def _comparison_query_handle(self, attr, v, operator):
if v.startswith(">="):
@ -126,21 +168,50 @@ class Search(object):
}
})
def _match_query_handle(self, attr, v, operator):
def _match_query_handle(self, attr, v, operator, is_not):
if "*" in v:
self._operator2query(operator).append({
"wildcard": {
attr: v.lower() if isinstance(v, six.string_types) else v
}
})
if is_not:
self._operator2query(operator).append({
"bool": {
"must_not": [
{
"wildcard": {
attr: v.lower() if isinstance(v, six.string_types) else v
}
}
]
}
})
else:
self._operator2query(operator).append({
"wildcard": {
attr: v.lower() if isinstance(v, six.string_types) else v
}
})
else:
if attr == "ci_type" and v.isdigit():
attr = "type_id"
self._operator2query(operator).append({
"term": {
attr: v.lower() if isinstance(v, six.string_types) else v
}
})
if is_not:
self._operator2query(operator).append({
"bool": {
"must_not": [
{
"term": {
attr: v.lower() if isinstance(v, six.string_types) else v
}
}
]
}
})
else:
self._operator2query(operator).append({
"term": {
attr: v.lower() if isinstance(v, six.string_types) else v
}
})
def __query_build_by_field(self, queries):
@ -150,21 +221,23 @@ class Search(object):
v = ":".join(q.split(":")[1:]).strip()
field_name, field_type, operator = self._attr_name_proc(k)
if field_name:
is_not = True if operator == "|~" else False
# in query
if v.startswith("(") and v.endswith(")"):
self._in_query_handle(field_name, v)
self._in_query_handle(field_name, v, is_not)
# range query
elif v.startswith("[") and v.endswith("]") and "_TO_" in v:
self._range_query_handle(field_name, v, operator)
self._range_query_handle(field_name, v, operator, is_not)
# comparison query
elif v.startswith(">=") or v.startswith("<=") or v.startswith(">") or v.startswith("<"):
self._comparison_query_handle(field_name, v, operator)
else:
self._match_query_handle(field_name, v, operator)
self._match_query_handle(field_name, v, operator, is_not)
else:
raise SearchError("argument q format invalid: {0}".format(q))
raise SearchError(ErrFormat.argument_invalid.format("q"))
elif q:
raise SearchError("argument q format invalid: {0}".format(q))
raise SearchError(ErrFormat.argument_invalid.format("q"))
def _query_build_raw(self):
@ -191,7 +264,7 @@ class Search(object):
for field in self.facet_field:
attr = AttributeCache.get(field)
if not attr:
raise SearchError("Facet by <{0}> does not exist".format(field))
raise SearchError(ErrFormat.attribute_not_found(field))
aggregations['aggs'].update({
field: {
"terms": {
@ -222,7 +295,7 @@ class Search(object):
attr = AttributeCache.get(field)
if not attr:
raise SearchError("Sort by <{0}> does not exist".format(field))
raise SearchError(ErrFormat.attribute_not_found.format(field))
sort_by = "{0}.keyword".format(field) \
if attr.value_type not in (ValueTypeEnum.INT, ValueTypeEnum.FLOAT) else field
@ -242,7 +315,7 @@ class Search(object):
numfound, cis, facet = self._query_build_raw()
except Exception as e:
current_app.logger.error(str(e))
raise SearchError("unknown search error")
raise SearchError(ErrFormat.unknown_search_error)
total = len(cis)

70
cmdb-api/api/lib/cmdb/search/ci_relation/search.py
View File

@ -2,13 +2,16 @@
import json
from collections import Counter
from flask import abort
from flask import current_app
from api.extensions import rd
from api.lib.cmdb.ci import CIRelationManager
from api.lib.cmdb.ci_type import CITypeRelationManager
from api.lib.cmdb.const import REDIS_PREFIX_CI_RELATION
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.cmdb.search.ci.db.search import Search as SearchFromDB
from api.lib.cmdb.search.ci.es.search import Search as SearchFromES
from api.models.cmdb import CI
@ -22,7 +25,8 @@ class Search(object):
facet_field=None,
page=1,
count=None,
sort=None):
sort=None,
reverse=False):
self.orig_query = query
self.fl = fl
self.facet_field = facet_field
@ -32,20 +36,35 @@ class Search(object):
self.root_id = root_id
self.level = level
self.reverse = reverse
def _get_ids(self):
merge_ids = []
ids = [self.root_id] if not isinstance(self.root_id, list) else self.root_id
for level in range(1, sorted(self.level)[-1] + 1):
_tmp = list(map(lambda x: list(json.loads(x).keys()),
filter(lambda x: x is not None, rd.get(ids, REDIS_PREFIX_CI_RELATION) or [])))
ids = [j for i in _tmp for j in i]
if level in self.level:
merge_ids.extend(ids)
return merge_ids
def _get_reverse_ids(self):
merge_ids = []
ids = [self.root_id] if not isinstance(self.root_id, list) else self.root_id
for level in range(1, sorted(self.level)[-1] + 1):
ids = CIRelationManager.get_ancestor_ids(ids, 1)
if level in self.level:
merge_ids.extend(ids)
return merge_ids
def search(self):
ids = [self.root_id] if not isinstance(self.root_id, list) else self.root_id
cis = [CI.get_by_id(_id) or abort(404, "CI <{0}> does not exist".format(_id)) for _id in ids]
cis = [CI.get_by_id(_id) or abort(404, ErrFormat.ci_not_found.format("id={}".format(_id))) for _id in ids]
merge_ids = []
for level in self.level:
ids = [self.root_id] if not isinstance(self.root_id, list) else self.root_id
for _ in range(0, level):
_tmp = list(map(lambda x: list(json.loads(x).keys()),
filter(lambda x: x is not None, rd.get(ids, REDIS_PREFIX_CI_RELATION) or [])))
ids = [j for i in _tmp for j in i]
merge_ids.extend(ids)
merge_ids = self._get_ids() if not self.reverse else self._get_reverse_ids()
if not self.orig_query or ("_type:" not in self.orig_query
and "type_id:" not in self.orig_query
@ -53,7 +72,10 @@ class Search(object):
type_ids = []
for level in self.level:
for ci in cis:
type_ids.extend(CITypeRelationManager.get_child_type_ids(ci.type_id, level))
if not self.reverse:
type_ids.extend(CITypeRelationManager.get_child_type_ids(ci.type_id, level))
else:
type_ids.extend(CITypeRelationManager.get_parent_type_ids(ci.type_id, level))
type_ids = list(set(type_ids))
if self.orig_query:
self.orig_query = "_type:({0}),{1}".format(";".join(list(map(str, type_ids))), self.orig_query)
@ -86,24 +108,30 @@ class Search(object):
ids = [self.root_id] if not isinstance(self.root_id, list) else self.root_id
for l in range(0, int(self.level)):
if not l:
_tmp = list(map(lambda x: list(json.loads(x).keys()),
_tmp = list(map(lambda x: list(json.loads(x).items()),
[i or '{}' for i in rd.get(ids, REDIS_PREFIX_CI_RELATION) or []]))
else:
for idx, i in enumerate(_tmp):
if i:
for idx, item in enumerate(_tmp):
if item:
if type_ids and l == self.level - 1:
__tmp = list(
map(lambda x: list({_id: 1 for _id, type_id in json.loads(x).items()
if type_id in type_ids}.keys()),
map(lambda x: [(_id, type_id) for _id, type_id in json.loads(x).items()
if type_id in type_ids],
filter(lambda x: x is not None,
rd.get(i, REDIS_PREFIX_CI_RELATION) or [])))
rd.get([i[0] for i in item], REDIS_PREFIX_CI_RELATION) or [])))
else:
__tmp = list(map(lambda x: list(json.loads(x).keys()),
__tmp = list(map(lambda x: list(json.loads(x).items()),
filter(lambda x: x is not None,
rd.get(i, REDIS_PREFIX_CI_RELATION) or [])))
rd.get([i[0] for i in item], REDIS_PREFIX_CI_RELATION) or [])))
_tmp[idx] = [j for i in __tmp for j in i]
else:
_tmp[idx] = []
return {_id: len(_tmp[idx]) for idx, _id in enumerate(ids)}
result = {str(_id): len(_tmp[idx]) for idx, _id in enumerate(ids)}
result.update(
detail={str(_id): dict(Counter([i[1] for i in _tmp[idx]]).items()) for idx, _id in enumerate(ids)})
return result

20
cmdb-api/api/lib/cmdb/utils.py
View File

@ -52,7 +52,7 @@ class ValueTypeMap(object):
ValueTypeEnum.FLOAT: float,
ValueTypeEnum.TEXT: lambda x: x.decode() if not isinstance(x, six.string_types) else x,
ValueTypeEnum.TIME: lambda x: x.decode() if not isinstance(x, six.string_types) else x,
ValueTypeEnum.DATE: lambda x: x.decode() if not isinstance(x, six.string_types) else x,
ValueTypeEnum.DATE: lambda x: (x.decode() if not isinstance(x, six.string_types) else x).split()[0],
ValueTypeEnum.DATETIME: lambda x: x.decode() if not isinstance(x, six.string_types) else x,
ValueTypeEnum.JSON: lambda x: json.loads(x) if isinstance(x, six.string_types) and x else x,
}
@ -109,17 +109,25 @@ class ValueTypeMap(object):
class TableMap(object):
def __init__(self, attr_name=None):
def __init__(self, attr_name=None, attr=None, is_index=None):
self.attr_name = attr_name
self.attr = attr
self.is_index = is_index
@property
def table(self):
attr = AttributeCache.get(self.attr_name)
i = "index_{0}".format(attr.value_type) if attr.is_index else attr.value_type
attr = AttributeCache.get(self.attr_name) if not self.attr else self.attr
if self.is_index is None:
self.is_index = attr.is_index
i = "index_{0}".format(attr.value_type) if self.is_index else attr.value_type
return ValueTypeMap.table.get(i)
@property
def table_name(self):
attr = AttributeCache.get(self.attr_name)
i = "index_{0}".format(attr.value_type) if attr.is_index else attr.value_type
attr = AttributeCache.get(self.attr_name) if not self.attr else self.attr
if self.is_index is None:
self.is_index = attr.is_index
i = "index_{0}".format(attr.value_type) if self.is_index else attr.value_type
return ValueTypeMap.table_name.get(i)

287
cmdb-api/api/lib/cmdb/value.py
View File

@ -3,7 +3,16 @@
from __future__ import unicode_literals
import copy
import imp
import os
import tempfile
import jinja2
from flask import abort
from flask import current_app
from jinja2schema import infer
from jinja2schema import to_json_schema
from api.extensions import db
from api.lib.cmdb.attribute import AttributeManager
@ -13,9 +22,11 @@ from api.lib.cmdb.const import ExistPolicy
from api.lib.cmdb.const import OperateType
from api.lib.cmdb.const import ValueTypeEnum
from api.lib.cmdb.history import AttributeHistoryManger
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.cmdb.utils import TableMap
from api.lib.cmdb.utils import ValueTypeMap
from api.lib.utils import handle_arg_list
from api.models.cmdb import CI
class AttributeValueManager(object):
@ -50,7 +61,7 @@ class AttributeValueManager(object):
if not attr:
continue
value_table = TableMap(attr_name=attr.name).table
value_table = TableMap(attr=attr).table
rs = value_table.get_by(ci_id=ci_id,
attr_id=attr.id,
use_master=use_master,
@ -64,6 +75,7 @@ class AttributeValueManager(object):
if unique_key is not None and attr.id == unique_key.id and rs:
res['unique'] = unique_key.name
res['unique_alias'] = unique_key.alias
return res
@ -76,48 +88,213 @@ class AttributeValueManager(object):
v = deserialize(value)
return v
except ValueError:
return abort(400, "attribute value <{0}> is invalid".format(value))
return abort(400, ErrFormat.attribute_value_invalid.format(value))
@staticmethod
def __check_is_choice(attr, value_type, value):
choice_values = AttributeManager.get_choice_values(attr.id, value_type)
if value not in choice_values:
return abort(400, "{0} does not existed in choice values".format(value))
choice_values = AttributeManager.get_choice_values(attr.id, value_type, attr.choice_web_hook)
if str(value) not in list(map(str, [i[0] for i in choice_values])):
return abort(400, ErrFormat.not_in_choice_values.format(value))
@staticmethod
def __check_is_unique(value_table, attr, ci_id, value):
existed = db.session.query(value_table.attr_id).filter(
def __check_is_unique(value_table, attr, ci_id, type_id, value):
existed = db.session.query(value_table.attr_id).join(CI, CI.id == value_table.ci_id).filter(
CI.type_id == type_id).filter(
value_table.attr_id == attr.id).filter(value_table.deleted.is_(False)).filter(
value_table.value == value).filter(value_table.ci_id != ci_id).first()
existed and abort(400, "attribute <{0}> value {1} must be unique".format(attr.alias, value))
existed and abort(400, ErrFormat.attribute_value_unique_required.format(attr.alias, value))
@staticmethod
def __check_is_required(type_id, attr, value):
type_attr = CITypeAttributeCache.get(type_id, attr.id)
def __check_is_required(type_id, attr, value, type_attr=None):
type_attr = type_attr or CITypeAttributeCache.get(type_id, attr.id)
if type_attr and type_attr.is_required and not value and value != 0:
return abort(400, "attribute <{0}> value is required".format(attr.alias))
return abort(400, ErrFormat.attribute_value_required.format(attr.alias))
def _validate(self, attr, value, value_table, ci):
def _validate(self, attr, value, value_table, ci=None, type_id=None, ci_id=None, type_attr=None):
ci = ci or {}
v = self.__deserialize_value(attr.value_type, value)
attr.is_choice and value and self.__check_is_choice(attr, attr.value_type, v)
attr.is_unique and self.__check_is_unique(value_table, attr, ci.id, v)
attr.is_unique and self.__check_is_unique(
value_table, attr, ci and ci.id or ci_id, ci and ci.type_id or type_id, v)
self.__check_is_required(ci.type_id, attr, v)
self.__check_is_required(ci and ci.type_id or type_id, attr, v, type_attr=type_attr)
if v == "" and attr.value_type not in (ValueTypeEnum.TEXT,):
v = None
return v
@staticmethod
def _write_change(ci_id, attr_id, operate_type, old, new):
AttributeHistoryManger.add(ci_id, [(attr_id, operate_type, old, new)])
def _write_change(ci_id, attr_id, operate_type, old, new, record_id, type_id):
return AttributeHistoryManger.add(record_id, ci_id, [(attr_id, operate_type, old, new)], type_id)
def create_or_update_attr_value(self, key, value, ci, _no_attribute_policy=ExistPolicy.IGNORE):
@staticmethod
def _write_change2(changed):
record_id = None
for ci_id, attr_id, operate_type, old, new, type_id in changed:
record_id = AttributeHistoryManger.add(record_id, ci_id, [(attr_id, operate_type, old, new)], type_id,
commit=False, flush=False)
try:
db.session.commit()
except Exception as e:
current_app.logger.error("write change failed: {}".format(str(e)))
return record_id
@staticmethod
def __compute_attr_value_from_expr(expr, ci_dict):
t = jinja2.Template(expr).render(ci_dict)
try:
return eval(t)
except Exception as e:
current_app.logger.warning(str(e))
return t
@staticmethod
def __compute_attr_value_from_script(script, ci_dict):
script = jinja2.Template(script).render(ci_dict)
script_f = tempfile.NamedTemporaryFile(delete=False, suffix=".py")
script_f.write(script.encode('utf-8'))
script_f.close()
try:
path = script_f.name
dir_name, name = os.path.dirname(path), os.path.basename(path)[:-3]
fp, path, desc = imp.find_module(name, [dir_name])
mod = imp.load_module(name, fp, path, desc)
if hasattr(mod, 'computed'):
return mod.computed()
except Exception as e:
current_app.logger.error(str(e))
finally:
os.remove(script_f.name)
@staticmethod
def _jinja2_parse(content):
schema = to_json_schema(infer(content))
return [var for var in schema.get("properties")]
def _compute_attr_value(self, attr, payload, ci):
attrs = self._jinja2_parse(attr['compute_expr']) if attr.get('compute_expr') else \
self._jinja2_parse(attr['compute_script'])
not_existed = [i for i in attrs if i not in payload]
if ci is not None:
payload.update(self.get_attr_values(not_existed, ci.id))
if attr['compute_expr']:
return self.__compute_attr_value_from_expr(attr['compute_expr'], payload)
elif attr['compute_script']:
return self.__compute_attr_value_from_script(attr['compute_script'], payload)
def handle_ci_compute_attributes(self, ci_dict, computed_attrs, ci):
payload = copy.deepcopy(ci_dict)
for attr in computed_attrs:
computed_value = self._compute_attr_value(attr, payload, ci)
if computed_value is not None:
ci_dict[attr['name']] = computed_value
def valid_attr_value(self, ci_dict, type_id, ci_id, name2attr, alias2attr=None, ci_attr2type_attr=None):
key2attr = dict()
alias2attr = alias2attr or {}
ci_attr2type_attr = ci_attr2type_attr or {}
for key, value in ci_dict.items():
attr = name2attr.get(key) or alias2attr.get(key)
key2attr[key] = attr
value_table = TableMap(attr=attr).table
try:
if attr.is_list:
value_list = [self._validate(attr, i, value_table, ci=None, type_id=type_id, ci_id=ci_id,
type_attr=ci_attr2type_attr.get(attr.id))
for i in handle_arg_list(value)]
ci_dict[key] = value_list
if not value_list:
self.__check_is_required(type_id, attr, '')
else:
value = self._validate(attr, value, value_table, ci=None, type_id=type_id, ci_id=ci_id,
type_attr=ci_attr2type_attr.get(attr.id))
ci_dict[key] = value
except Exception as e:
current_app.logger.warning(str(e))
return abort(400, ErrFormat.attribute_value_invalid2.format(
"{}({})".format(attr.alias, attr.name), value))
return key2attr
def create_or_update_attr_value2(self, ci, ci_dict, key2attr):
"""
add or update attribute value, then write history
:param ci: instance object
:param ci_dict: attribute dict
:param key2attr: attr key to attr
:return:
"""
changed = []
for key, value in ci_dict.items():
attr = key2attr.get(key)
if not attr:
continue # not be here
value_table = TableMap(attr=attr).table
if attr.is_list:
existed_attrs = value_table.get_by(attr_id=attr.id, ci_id=ci.id, to_dict=False)
existed_values = [i.value for i in existed_attrs]
added = set(value) - set(existed_values)
deleted = set(existed_values) - set(value)
for v in added:
value_table.create(ci_id=ci.id, attr_id=attr.id, value=v, flush=False, commit=False)
changed.append((ci.id, attr.id, OperateType.ADD, None, v, ci.type_id))
for v in deleted:
existed_attr = existed_attrs[existed_values.index(v)]
existed_attr.delete(flush=False, commit=False)
changed.append((ci.id, attr.id, OperateType.DELETE, v, None, ci.type_id))
else:
existed_attr = value_table.get_by(attr_id=attr.id, ci_id=ci.id, first=True, to_dict=False)
existed_value = existed_attr and existed_attr.value
if existed_value is None and value is not None:
value_table.create(ci_id=ci.id, attr_id=attr.id, value=value, flush=False, commit=False)
changed.append((ci.id, attr.id, OperateType.ADD, None, value, ci.type_id))
else:
if existed_value != value:
if value is None:
existed_attr.delete(flush=False, commit=False)
else:
existed_attr.update(value=value, flush=False, commit=False)
changed.append((ci.id, attr.id, OperateType.UPDATE, existed_value, value, ci.type_id))
try:
db.session.commit()
except Exception as e:
db.session.rollback()
current_app.logger.warning(str(e))
return abort(400, ErrFormat.attribute_value_unknown_error.format(str(e)))
return self._write_change2(changed)
def create_or_update_attr_value(self, key, value, ci, _no_attribute_policy=ExistPolicy.IGNORE, record_id=None):
"""
add or update attribute value, then write history
:param key: id, name or alias
:param value:
:param ci: instance object
:param _no_attribute_policy: ignore or reject
:param record_id: op record
:return:
"""
attr = self._get_attr(key)
@ -125,53 +302,55 @@ class AttributeValueManager(object):
if _no_attribute_policy == ExistPolicy.IGNORE:
return
if _no_attribute_policy == ExistPolicy.REJECT:
return abort(400, 'attribute {0} does not exist'.format(key))
return abort(400, ErrFormat.attribute_not_found.format(key))
value_table = TableMap(attr_name=attr.name).table
value_table = TableMap(attr=attr).table
if attr.is_list:
value_list = [self._validate(attr, i, value_table, ci) for i in handle_arg_list(value)]
if not value_list:
self.__check_is_required(ci.type_id, attr, '')
try:
if attr.is_list:
value_list = [self._validate(attr, i, value_table, ci) for i in handle_arg_list(value)]
if not value_list:
self.__check_is_required(ci.type_id, attr, '')
existed_attrs = value_table.get_by(attr_id=attr.id,
ci_id=ci.id,
to_dict=False)
existed_values = [i.value for i in existed_attrs]
added = set(value_list) - set(existed_values)
deleted = set(existed_values) - set(value_list)
for v in added:
value_table.create(ci_id=ci.id, attr_id=attr.id, value=v)
self._write_change(ci.id, attr.id, OperateType.ADD, None, v)
existed_attrs = value_table.get_by(attr_id=attr.id, ci_id=ci.id, to_dict=False)
existed_values = [i.value for i in existed_attrs]
added = set(value_list) - set(existed_values)
deleted = set(existed_values) - set(value_list)
for v in added:
value_table.create(ci_id=ci.id, attr_id=attr.id, value=v)
record_id = self._write_change(ci.id, attr.id, OperateType.ADD, None, v, record_id, ci.type_id)
for v in deleted:
existed_attr = existed_attrs[existed_values.index(v)]
existed_attr.delete()
self._write_change(ci.id, attr.id, OperateType.DELETE, v, None)
else:
value = self._validate(attr, value, value_table, ci)
existed_attr = value_table.get_by(attr_id=attr.id,
ci_id=ci.id,
first=True,
to_dict=False)
existed_value = existed_attr and existed_attr.value
if existed_value is None:
value_table.create(ci_id=ci.id, attr_id=attr.id, value=value)
self._write_change(ci.id, attr.id, OperateType.ADD, None, value)
for v in deleted:
existed_attr = existed_attrs[existed_values.index(v)]
existed_attr.delete()
record_id = self._write_change(ci.id, attr.id, OperateType.DELETE, v, None, record_id, ci.type_id)
else:
if existed_value != value:
if value != 0 and not value and attr.value_type != ValueTypeEnum.TEXT:
existed_attr.delete()
else:
existed_attr.update(value=value)
value = self._validate(attr, value, value_table, ci)
existed_attr = value_table.get_by(attr_id=attr.id, ci_id=ci.id, first=True, to_dict=False)
existed_value = existed_attr and existed_attr.value
if existed_value is None and value is not None:
value_table.create(ci_id=ci.id, attr_id=attr.id, value=value)
self._write_change(ci.id, attr.id, OperateType.UPDATE, existed_value, value)
record_id = self._write_change(ci.id, attr.id, OperateType.ADD, None, value, record_id, ci.type_id)
else:
if existed_value != value:
if value is None:
existed_attr.delete()
else:
existed_attr.update(value=value)
record_id = self._write_change(ci.id, attr.id, OperateType.UPDATE,
existed_value, value, record_id, ci.type_id)
return record_id
except Exception as e:
current_app.logger.warning(str(e))
return abort(400, ErrFormat.attribute_value_invalid2.format("{}({})".format(attr.alias, attr.name), value))
@staticmethod
def delete_attr_value(attr_id, ci_id):
attr = AttributeCache.get(attr_id)
if attr is not None:
value_table = TableMap(attr_name=attr.name).table
value_table = TableMap(attr=attr).table
for item in value_table.get_by(attr_id=attr.id, ci_id=ci_id, to_dict=False):
item.delete()

0
cmdb-api/api/lib/common_setting/__init__.py Normal file
View File

95
cmdb-api/api/lib/common_setting/acl.py Normal file
View File

@ -0,0 +1,95 @@
# -*- coding:utf-8 -*-
from flask import abort
from flask import current_app
from api.lib.common_setting.resp_format import ErrFormat
from api.lib.perm.acl.cache import RoleCache, AppCache
from api.lib.perm.acl.role import RoleCRUD, RoleRelationCRUD
from api.lib.perm.acl.user import UserCRUD
class ACLManager(object):
def __init__(self, app_name='acl', uid=None):
self.log = current_app.logger
self.app_name = app_name
self.uid = uid
@staticmethod
def get_all_users():
try:
numfound, users = UserCRUD.search(None, 1, 999999)
users = [i.to_dict() for i in users]
for u in users:
u.pop('password', None)
u.pop('key', None)
u.pop('secret', None)
return users
except Exception as e:
current_app.logger.error(str(e))
raise Exception(ErrFormat.acl_get_all_users_failed.format(str(e)))
@staticmethod
def create_user(payload):
user = UserCRUD.add(**payload)
return user.to_dict()
@staticmethod
def edit_user(uid, payload):
user = UserCRUD.update(uid, **payload)
return user.to_dict()
def get_all_roles(self):
numfound, roles = RoleCRUD.search(
None, self.app_name, 1, 999999, True, True, False)
return roles
def remove_user_from_role(self, user_rid, payload):
app_id = self.app_name
app = AppCache.get(app_id)
if app and app.name == "acl":
app_id = None # global
RoleRelationCRUD.delete2(
payload.get('parent_id'), user_rid, app_id)
return dict(
message="success"
)
def add_user_to_role(self, role_id, payload):
app_id = self.app_name
app = AppCache.get(self.app_name)
if app and app.name == "acl":
app_id = None
role = RoleCache.get(role_id)
res = RoleRelationCRUD.add(
role, role_id, payload['child_ids'], app_id)
return res
@staticmethod
def create_role(payload):
payload['is_app_admin'] = payload.get('is_app_admin', False)
role = RoleCRUD.add_role(**payload)
return role.to_dict()
@staticmethod
def edit_role(_id, payload):
role = RoleCRUD.update_role(_id, **payload)
return role.to_dict()
@staticmethod
def delete_role(_id, payload):
RoleCRUD.delete_role(_id)
return dict(rid=_id)
def get_user_info(self, username):
from api.lib.perm.acl.acl import ACLManager as ACL
user_info = ACL().get_user_info(username, self.app_name)
result = dict(name=user_info.get('nickname') or username,
username=user_info.get('username') or username,
email=user_info.get('email'),
uid=user_info.get('uid'),
rid=user_info.get('rid'),
role=dict(permissions=user_info.get('parents')),
avatar=user_info.get('avatar'))
return result

24
cmdb-api/api/lib/common_setting/company_info.py Normal file
View File

@ -0,0 +1,24 @@
# -*- coding:utf-8 -*-
from api.models.common_setting import CompanyInfo
class CompanyInfoCRUD(object):
@staticmethod
def get():
return CompanyInfo.get_by(first=True) or {}
@staticmethod
def create(**kwargs):
return CompanyInfo.create(**kwargs)
@staticmethod
def update(_id, **kwargs):
kwargs.pop('id', None)
existed = CompanyInfo.get_by_id(_id)
if not existed:
return CompanyInfoCRUD.create(**kwargs)
else:
existed = existed.update(**kwargs)
return existed

14
cmdb-api/api/lib/common_setting/const.py Normal file
View File

@ -0,0 +1,14 @@
from api.lib.common_setting.utils import BaseEnum
COMMON_SETTING_QUEUE = "common_setting_async"
class OperatorType(BaseEnum):
EQUAL = 1 # 等于
NOT_EQUAL = 2 # 不等于
IN = 3 # 包含
NOT_IN = 4 # 不包含
GREATER_THAN = 5 # 大于
LESS_THAN = 6 # 小于
IS_EMPTY = 7 # 为空
IS_NOT_EMPTY = 8 # 不为空

435
cmdb-api/api/lib/common_setting/department.py Normal file
View File

@ -0,0 +1,435 @@
# -*- coding:utf-8 -*-
from flask import abort
from treelib import Tree
from wtforms import Form
from wtforms import IntegerField
from wtforms import StringField
from wtforms import validators
from api.lib.common_setting.resp_format import ErrFormat
from api.lib.common_setting.utils import get_df_from_read_sql
from api.lib.perm.acl.role import RoleCRUD
from api.models.common_setting import Department, Employee
sub_departments_column_name = 'sub_departments'
def drop_ts_column(df):
columns = list(df.columns)
remove_columns = []
for column in ['created_at', 'updated_at', 'deleted_at', 'last_login']:
targets = list(filter(lambda c: c.startswith(column), columns))
if targets:
remove_columns.extend(targets)
remove_columns = list(set(remove_columns))
return df.drop(remove_columns, axis=1) if len(remove_columns) > 0 else df
def get_department_df():
criterion = [
Department.deleted == 0,
]
query = Department.query.filter(
*criterion
)
df = get_df_from_read_sql(query)
if df.empty:
return
return drop_ts_column(df)
def get_all_employee_df(block=0):
criterion = [
Employee.deleted == 0,
]
if block >= 0:
criterion.append(
Employee.block == block
)
entities = [getattr(Employee, c) for c in Employee.get_columns(
).keys() if c not in ['deleted', 'deleted_at']]
query = Employee.query.with_entities(
*entities
).filter(
*criterion
)
df = get_df_from_read_sql(query)
if df.empty:
return df
return drop_ts_column(df)
class DepartmentTree(object):
def __init__(self, append_employee=False, block=-1):
self.append_employee = append_employee
self.block = block
self.d_df = get_department_df()
self.employee_df = get_all_employee_df(
block) if append_employee else None
def prepare(self):
pass
def get_employees_by_d_id(self, d_id):
_df = self.employee_df[
self.employee_df['department_id'].eq(d_id)
].sort_values(by=['direct_supervisor_id'], ascending=True)
if _df.empty:
return []
if self.block != -1:
_df = _df[
_df['block'].eq(self.block)
]
return _df.to_dict('records')
def get_tree_departments(self):
# 一级部门
top_df = self.d_df[self.d_df['department_parent_id'].eq(-1)]
if top_df.empty:
return []
d_list = []
for index in top_df.index:
top_d = top_df.loc[index].to_dict()
department_id = top_d['department_id']
# 检查 department_id 是否作为其他部门的 parent
sub_df = self.d_df[
self.d_df['department_parent_id'].eq(department_id)
].sort_values(by=['sort_value'], ascending=True)
employees = []
if self.append_employee:
# 要包含员工
employees = self.get_employees_by_d_id(department_id)
top_d['employees'] = employees
if sub_df.empty:
top_d[sub_departments_column_name] = []
d_list.append(top_d)
continue
self.parse_sub_department(sub_df, top_d)
d_list.append(top_d)
return d_list
def get_all_departments(self, is_tree=1):
if self.d_df.empty:
return []
if is_tree != 1:
return self.d_df.to_dict('records')
return self.get_tree_departments()
def parse_sub_department(self, df, top_d):
sub_departments = []
for s_index in df.index:
d = df.loc[s_index].to_dict()
sub_df = self.d_df[
self.d_df['department_parent_id'].eq(
df.at[s_index, 'department_id'])
].sort_values(by=['sort_value'], ascending=True)
employees = []
if self.append_employee:
# 要包含员工
employees = self.get_employees_by_d_id(
df.at[s_index, 'department_id'])
d['employees'] = employees
if sub_df.empty:
d[sub_departments_column_name] = []
sub_departments.append(d)
continue
self.parse_sub_department(sub_df, d)
sub_departments.append(d)
top_d[sub_departments_column_name] = sub_departments
class DepartmentForm(Form):
department_name = StringField(validators=[
validators.DataRequired(message="部门名称不能为空"),
validators.Length(max=255),
])
department_director_id = IntegerField(validators=[], default=0)
department_parent_id = IntegerField(validators=[], default=1)
class DepartmentCRUD(object):
@staticmethod
def add(**kwargs):
DepartmentCRUD.check_department_name_unique(kwargs['department_name'])
department_parent_id = kwargs.get('department_parent_id', 0)
DepartmentCRUD.check_department_parent_id(department_parent_id)
DepartmentCRUD.check_department_parent_id_allow(
-1, department_parent_id)
try:
role = RoleCRUD.add_role(name=kwargs['department_name'])
except Exception as e:
return abort(400, ErrFormat.acl_add_role_failed.format(str(e)))
kwargs['acl_rid'] = role.id
try:
db_department = Department.create(
**kwargs
)
except Exception as e:
return abort(400, str(e))
return db_department
@staticmethod
def check_department_parent_id_allow(d_id, department_parent_id):
if department_parent_id == 0:
return
# 检查 department_parent_id 是否在许可范围内
allow_p_d_id_list = DepartmentCRUD.get_allow_parent_d_id_by(d_id)
target = list(
filter(lambda d: d['department_id'] == department_parent_id, allow_p_d_id_list))
if len(target) == 0:
try:
d = Department.get_by(
first=True, to_dict=False, department_id=department_parent_id)
name = d.department_name if d else ErrFormat.department_id_not_found.format(department_parent_id)
except Exception as e:
name = ErrFormat.department_id_not_found.format(department_parent_id)
abort(400, ErrFormat.cannot_to_be_parent_department.format(name))
@staticmethod
def check_department_parent_id(department_parent_id):
if int(department_parent_id) < 0:
abort(400, ErrFormat.parent_department_id_must_more_than_zero)
@staticmethod
def check_department_name_unique(name, _id=0):
criterion = [
Department.department_name == name,
Department.deleted == 0,
]
if _id > 0:
criterion.append(
Department.department_id != _id
)
res = Department.query.filter(
*criterion
).all()
res and abort(
400, ErrFormat.department_name_already_exists.format(name)
)
@staticmethod
def edit(_id, **kwargs):
DepartmentCRUD.check_department_name_unique(
kwargs['department_name'], _id)
kwargs.pop('department_id', None)
existed = Department.get_by(
first=True, department_id=_id, to_dict=False)
if not existed:
abort(404, ErrFormat.department_id_not_found.format(_id))
department_parent_id = kwargs.get('department_parent_id', 0)
DepartmentCRUD.check_department_parent_id(department_parent_id)
if department_parent_id > 0:
DepartmentCRUD.check_department_parent_id_allow(
_id, department_parent_id)
try:
RoleCRUD.update_role(
existed.acl_rid, name=kwargs['department_name'])
except Exception as e:
return abort(400, ErrFormat.acl_update_role_failed.format(str(e)))
try:
existed.update(**kwargs)
except Exception as e:
return abort(400, str(e))
@staticmethod
def delete(_id):
existed = Department.get_by(
first=True, department_id=_id, to_dict=False)
if not existed:
abort(404, ErrFormat.department_id_not_found.format(_id))
try:
RoleCRUD.delete_role(existed.acl_rid)
except Exception as e:
pass
return existed.soft_delete()
@staticmethod
def get_allow_parent_d_id_by(department_id):
"""
获取可以成为 department_id 的 department_parent_id 的 list
"""
tree_list = DepartmentCRUD.get_department_tree_list()
allow_d_id_list = []
for tree in tree_list:
if department_id > 0:
try:
tree.remove_subtree(department_id)
except Exception as e:
pass
[allow_d_id_list.append({'department_id': int(n.identifier), 'department_name': n.tag}) for n in
tree.all_nodes()]
return allow_d_id_list
@staticmethod
def update_department_sort(department_list):
d_map = {d['id']: d['sort_value'] for d in department_list}
d_id = [d['id'] for d in department_list]
db_list = Department.query.filter(
Department.department_id.in_(d_id),
Department.deleted == 0
).all()
for existed in db_list:
existed.update(sort_value=d_map[existed.department_id])
return []
@staticmethod
def get_all_departments_with_employee(block):
return DepartmentTree(True, block).get_all_departments(1)
@staticmethod
def get_department_tree_list():
df = get_department_df()
if df.empty:
return []
# 一级部门
top_df = df[df['department_parent_id'].eq(-1)]
if top_df.empty:
return []
tree_list = []
for index in top_df.index:
tree = Tree()
identifier_root = top_df.at[index, 'department_id']
tree.create_node(
top_df.at[index, 'department_name'],
identifier_root
)
# 检查 department_id 是否作为其他部门的 parent
sub_df = df[
df['department_parent_id'].eq(identifier_root)
]
if sub_df.empty:
tree_list.append(tree)
continue
DepartmentCRUD.parse_sub_department_node(
sub_df, df, tree, identifier_root)
tree_list.append(tree)
return tree_list
@staticmethod
def parse_sub_department_node(df, all_df, tree, parent_id):
for s_index in df.index:
tree.create_node(
df.at[s_index, 'department_name'],
df.at[s_index, 'department_id'],
parent=parent_id
)
sub_df = all_df[
all_df['department_parent_id'].eq(
df.at[s_index, 'department_id'])
]
if sub_df.empty:
continue
DepartmentCRUD.parse_sub_department_node(
sub_df, all_df, tree, df.at[s_index, 'department_id'])
@staticmethod
def get_departments_and_ids(department_parent_id, block):
query = Department.query.filter(
Department.department_parent_id == department_parent_id,
Department.deleted == 0,
).order_by(Department.sort_value.asc())
df = get_df_from_read_sql(query)
if df.empty:
return [], []
tree_list = DepartmentCRUD.get_department_tree_list()
employee_df = get_all_employee_df(block)
department_id_list = list(df['department_id'].values)
query = Department.query.filter(
Department.department_parent_id.in_(department_id_list),
Department.deleted == 0,
).order_by(Department.sort_value.asc()).group_by(Department.department_id)
sub_df = get_df_from_read_sql(query)
if sub_df.empty:
df['has_sub'] = 0
def handle_row_employee_count(row):
return len(employee_df[employee_df['department_id'] == row['department_id']])
df['employee_count'] = df.apply(
lambda row: handle_row_employee_count(row), axis=1)
else:
sub_map = {d['department_parent_id']: 1 for d in sub_df.to_dict('records')}
def handle_row(row):
d_ids = DepartmentCRUD.get_department_id_list_by_root(
row['department_id'], tree_list)
row['employee_count'] = len(
employee_df[employee_df['department_id'].isin(d_ids)])
row['has_sub'] = sub_map.get(row['department_id'], 0)
return row
df = df.apply(lambda row: handle_row(row), axis=1)
return df.to_dict('records'), department_id_list
@staticmethod
def get_department_id_list_by_root(root_department_id, tree_list=None):
if tree_list is None:
tree_list = DepartmentCRUD.get_department_tree_list()
id_list = []
for tree in tree_list:
try:
tmp_tree = tree.subtree(root_department_id)
[id_list.append(int(n.identifier))
for n in tmp_tree.all_nodes()]
except Exception as e:
pass
return id_list

842
cmdb-api/api/lib/common_setting/employee.py Normal file
View File

@ -0,0 +1,842 @@
# -*- coding:utf-8 -*-
import traceback
from datetime import datetime
import pandas as pd
from flask import abort
from flask_login import current_user
from sqlalchemy import or_, literal_column, func, not_, and_
from werkzeug.datastructures import MultiDict
from wtforms import Form
from wtforms import IntegerField
from wtforms import StringField
from wtforms import validators
from api.extensions import db
from api.lib.common_setting.acl import ACLManager
from api.lib.common_setting.const import COMMON_SETTING_QUEUE, OperatorType
from api.lib.common_setting.resp_format import ErrFormat
from api.lib.common_setting.utils import get_df_from_read_sql
from api.models.common_setting import Employee, Department
def edit_acl_user(uid, **kwargs):
user_data = {column: kwargs.get(
column, '') for column in acl_user_columns if kwargs.get(column, '')}
if 'block' in kwargs:
user_data['block'] = kwargs.get('block')
try:
acl = ACLManager()
return acl.edit_user(uid, user_data)
except Exception as e:
abort(400, ErrFormat.acl_edit_user_failed.format(str(e)))
def get_block_value(value):
if value in ['False', 'false', '0', 0]:
value = False
else:
value = True
return value
def get_employee_list_by_direct_supervisor_id(direct_supervisor_id):
return Employee.get_by(direct_supervisor_id=direct_supervisor_id)
def get_department_list_by_director_id(director_id):
return Department.get_by(department_director_id=director_id)
def raise_exception(err):
raise Exception(err)
def check_department_director_id_or_direct_supervisor_id(_id):
get_employee_list_by_direct_supervisor_id(
_id) and raise_exception(ErrFormat.cannot_block_this_employee_is_other_direct_supervisor)
get_department_list_by_director_id(
_id) and raise_exception(ErrFormat.cannot_block_this_employee_is_department_manager)
class EmployeeCRUD(object):
@staticmethod
def get_employee_by_id(_id):
return Employee.get_by(
first=True, to_dict=False, deleted=0, employee_id=_id
) or abort(404, ErrFormat.employee_id_not_found.format(_id))
@staticmethod
def get_employee_by_uid_with_create(_uid):
"""
根据 uid 获取员工信息，不存在则创建
"""
try:
return EmployeeCRUD.get_employee_by_uid(_uid).to_dict()
except Exception as e:
if '不存在' not in str(e):
abort(400, str(e))
try:
acl = ACLManager('acl')
user_info = acl.get_user_info(_uid)
return EmployeeCRUD.check_acl_user_and_create(user_info)
except Exception as e:
abort(400, str(e))
@staticmethod
def get_employee_by_uid(_uid):
return Employee.get_by(
first=True, to_dict=False, deleted=0, acl_uid=_uid
) or abort(404, ErrFormat.acl_uid_not_found.format(_uid))
@staticmethod
def check_acl_user_and_create(user_info):
existed = Employee.get_by(
first=True, to_dict=False, username=user_info['username'])
if existed:
existed.update(
acl_uid=user_info['uid'],
)
return existed.to_dict()
# 创建员工
if not user_info.get('nickname', None):
user_info['nickname'] = user_info['name']
form = EmployeeAddForm(MultiDict(user_info))
data = form.data
data['password'] = ''
data['acl_uid'] = user_info['uid']
employee = CreateEmployee().create_single(**data)
return employee.to_dict()
@staticmethod
def add(**kwargs):
try:
return CreateEmployee().create_single(**kwargs)
except Exception as e:
abort(400, str(e))
@staticmethod
def update(_id, **kwargs):
EmployeeCRUD.check_email_unique(kwargs['email'], _id)
existed = EmployeeCRUD.get_employee_by_id(_id)
try:
edit_acl_user(existed.acl_uid, **kwargs)
for column in employee_pop_columns:
kwargs.pop(column, None)
new_department_id = kwargs.get('department_id', None)
e_list = []
if new_department_id is not None and new_department_id != existed.department_id:
e_list = [dict(
e_acl_rid=existed.acl_rid,
department_id=existed.department_id
)]
existed.update(**kwargs)
if len(e_list) > 0:
from api.tasks.common_setting import edit_employee_department_in_acl
# fixme: comment next line
# edit_employee_department_in_acl(e_list, new_department_id, current_user.uid)
edit_employee_department_in_acl.apply_async(
args=(e_list, new_department_id, current_user.uid),
queue=COMMON_SETTING_QUEUE
)
return existed
except Exception as e:
return abort(400, str(e))
@staticmethod
def edit_employee_by_uid(_uid, **kwargs):
existed = EmployeeCRUD.get_employee_by_uid(_uid)
try:
user = edit_acl_user(_uid, **kwargs)
for column in employee_pop_columns:
if kwargs.get(column):
kwargs.pop(column)
return existed.update(**kwargs)
except Exception as e:
return abort(400, str(e))
@staticmethod
def change_password_by_uid(_uid, password):
existed = EmployeeCRUD.get_employee_by_uid(_uid)
try:
user = edit_acl_user(_uid, password=password)
except Exception as e:
return abort(400, str(e))
@staticmethod
def get_all_position():
criterion = [
Employee.deleted == 0,
]
results = Employee.query.with_entities(
Employee.position_name
).filter(*criterion).group_by(
Employee.position_name
).order_by(
func.CONVERT(literal_column('position_name using gbk'))
).all()
return [item[0] for item in results if (item[0] is not None and item[0] != '')]
@staticmethod
def get_employee_count(block_status):
criterion = [
Employee.deleted == 0
]
if block_status >= 0:
criterion.append(
Employee.block == block_status
)
return Employee.query.filter(
*criterion
).count()
@staticmethod
def import_employee(employee_list):
return CreateEmployee().batch_create(employee_list)
@staticmethod
def get_export_employee_df(block_status):
criterion = [
Employee.deleted == 0
]
if block_status >= 0:
criterion.append(
Employee.block == block_status
)
query = Employee.query.with_entities(
Employee.employee_id,
Employee.nickname,
Employee.email,
Employee.sex,
Employee.mobile,
Employee.position_name,
Employee.last_login,
Employee.department_id,
Employee.direct_supervisor_id,
).filter(*criterion)
df = get_df_from_read_sql(query)
if df.empty:
return df
query = Department.query.filter(
*criterion
)
department_df = get_df_from_read_sql(query)
def find_name(row):
department_id = row['department_id']
_df = department_df[department_df['department_id']
== department_id]
row['department_name'] = '' if _df.empty else _df.iloc[0]['department_name']
direct_supervisor_id = row['direct_supervisor_id']
_df = df[df['employee_id'] == direct_supervisor_id]
row['nickname_direct_supervisor'] = '' if _df.empty else _df.iloc[0]['nickname']
if isinstance(row['last_login'], pd.Timestamp):
try:
row['last_login'] = str(row['last_login'])
except:
row['last_login'] = ''
else:
row['last_login'] = ''
return row
df = df.apply(find_name, axis=1)
df.drop(['department_id', 'direct_supervisor_id',
'employee_id'], axis=1, inplace=True)
return df
@staticmethod
def batch_employee(column_name, column_value, employee_id_list):
if not column_value:
abort(400, ErrFormat.value_is_required)
if column_name in ['password', 'block']:
return EmployeeCRUD.batch_edit_password_or_block_column(column_name, employee_id_list, column_value, True)
elif column_name in ['department_id']:
return EmployeeCRUD.batch_edit_employee_department(employee_id_list, column_value)
elif column_name in [
'direct_supervisor_id', 'position_name'
]:
return EmployeeCRUD.batch_edit_column(column_name, employee_id_list, column_value, False)
else:
abort(400, ErrFormat.column_name_not_support)
@staticmethod
def batch_edit_employee_department(employee_id_list, column_value):
err_list = []
employee_list = []
for _id in employee_id_list:
try:
existed = EmployeeCRUD.get_employee_by_id(_id)
employee = dict(
e_acl_rid=existed.acl_rid,
department_id=existed.department_id
)
employee_list.append(employee)
existed.update(department_id=column_value)
except Exception as e:
err_list.append({
'employee_id': _id,
'err': str(e),
})
from api.tasks.common_setting import edit_employee_department_in_acl
edit_employee_department_in_acl.apply_async(
args=(employee_list, column_value, current_user.uid),
queue=COMMON_SETTING_QUEUE
)
return err_list
@staticmethod
def batch_edit_password_or_block_column(column_name, employee_id_list, column_value, is_acl=False):
if column_name == 'block':
err_list = []
success_list = []
for _id in employee_id_list:
try:
employee = EmployeeCRUD.edit_employee_block_column(
_id, is_acl, **{column_name: column_value})
success_list.append(employee)
except Exception as e:
err_list.append({
'employee_id': _id,
'err': str(e),
})
return err_list
else:
return EmployeeCRUD.batch_edit_column(column_name, employee_id_list, column_value, is_acl)
@staticmethod
def batch_edit_column(column_name, employee_id_list, column_value, is_acl=False):
err_list = []
for _id in employee_id_list:
try:
EmployeeCRUD.edit_employee_single_column(
_id, is_acl, **{column_name: column_value})
except Exception as e:
err_list.append({
'employee_id': _id,
'err': str(e),
})
return err_list
@staticmethod
def edit_employee_single_column(_id, is_acl=False, **kwargs):
existed = EmployeeCRUD.get_employee_by_id(_id)
if is_acl:
return edit_acl_user(existed.acl_uid, **kwargs)
try:
for column in employee_pop_columns:
if kwargs.get(column):
kwargs.pop(column)
return existed.update(**kwargs)
except Exception as e:
return abort(400, str(e))
@staticmethod
def edit_employee_block_column(_id, is_acl=False, **kwargs):
existed = EmployeeCRUD.get_employee_by_id(_id)
value = get_block_value(kwargs.get('block'))
if value is True:
# 判断该用户是否为 部门负责人，或者员工的直接上级
check_department_director_id_or_direct_supervisor_id(_id)
if is_acl:
kwargs['block'] = value
edit_acl_user(existed.acl_uid, **kwargs)
data = existed.to_dict()
return data
@staticmethod
def check_email_unique(email, _id=0):
criterion = [
Employee.email == email,
Employee.deleted == 0,
]
if _id > 0:
criterion.append(
Employee.employee_id != _id
)
res = Employee.query.filter(
*criterion
).all()
if res:
err = ErrFormat.email_already_exists.format(email)
raise Exception(err)
@staticmethod
def get_employee_list_by_body(department_id, block_status, search='', order='', conditions=[], page=1,
page_size=10):
criterion = [
Employee.deleted == 0
]
if block_status >= 0:
criterion.append(
Employee.block == block_status
)
if len(search) > 0:
search_key = f"%{search}%"
criterion.append(
or_(
Employee.email.like(search_key),
Employee.username.like(search_key),
Employee.nickname.like(search_key)
)
)
if department_id > 0:
from api.lib.common_setting.department import DepartmentCRUD
department_id_list = DepartmentCRUD.get_department_id_list_by_root(
department_id)
criterion.append(
Employee.department_id.in_(department_id_list)
)
if conditions:
query = EmployeeCRUD.parse_condition_list_to_query(conditions).filter(
*criterion
)
else:
query = db.session.query(Employee, Department).outerjoin(Department).filter(
*criterion
)
if len(order) > 0:
query = EmployeeCRUD.format_query_sort(query, order)
pagination = query.paginate(page=page, per_page=page_size)
employees = []
for r in pagination.items:
d = r.Employee.to_dict()
d['department_name'] = r.Department.department_name
employees.append(d)
return {
'data_list': employees,
'page': page,
'page_size': page_size,
'total': pagination.total,
}
@staticmethod
def parse_condition_list_to_query(condition_list):
query = db.session.query(Employee, Department).outerjoin(Department)
query = EmployeeCRUD.get_query_by_conditions(query, condition_list)
return query
@staticmethod
def get_expr_by_condition(column, operator, value, relation):
"""
根据conditions返回expr: (and_list, or_list)
"""
attr = EmployeeCRUD.get_attr_by_column(column)
# 根据operator生成条件表达式
if operator == OperatorType.EQUAL:
expr = [attr == value]
elif operator == OperatorType.NOT_EQUAL:
expr = [attr != value]
elif operator == OperatorType.IN:
expr = [attr.like('%{}%'.format(value))]
elif operator == OperatorType.NOT_IN:
expr = [not_(attr.like('%{}%'.format(value)))]
elif operator == OperatorType.GREATER_THAN:
expr = [attr > value]
elif operator == OperatorType.LESS_THAN:
expr = [attr < value]
elif operator == OperatorType.IS_EMPTY:
if value:
abort(400, ErrFormat.query_column_none_keep_value_empty.format(column))
expr = [attr.is_(None)]
if column not in ["entry_date", "leave_date", "dfc_entry_date", "last_login"]:
expr += [attr == '']
expr = [or_(*expr)]
elif operator == OperatorType.IS_NOT_EMPTY:
if value:
abort(400, ErrFormat.query_column_none_keep_value_empty.format(column))
expr = [attr.isnot(None)]
if column not in ["last_login"]:
expr += [attr != '']
expr = [and_(*expr)]
else:
abort(400, ErrFormat.not_support_operator.format(operator))
# 根据relation生成复合条件
if relation == "&":
return expr, []
elif relation == "|":
return [], expr
else:
return abort(400, ErrFormat.not_support_relation.format(relation))
@staticmethod
def check_condition(column, operator, value, relation):
# 对于condition中column为空的，报错
if column is None or operator is None or relation is None:
return abort(400, ErrFormat.conditions_field_missing)
if value and column == "last_login":
try:
value = datetime.strptime(value, "%Y-%m-%d %H:%M:%S")
except Exception as e:
abort(400, ErrFormat.datetime_format_error.format(column))
@staticmethod
def get_attr_by_column(column):
if 'department' in column:
attr = Department.__dict__[column]
else:
attr = Employee.__dict__[column]
return attr
@staticmethod
def get_query_by_conditions(query, conditions):
and_list = []
or_list = []
for condition in conditions:
operator = condition.get("operator", None)
column = condition.get("column", None)
relation = condition.get("relation", None)
value = condition.get("value", None)
EmployeeCRUD.check_condition(column, operator, value, relation)
a, o = EmployeeCRUD.get_expr_by_condition(
column, operator, value, relation)
and_list += a
or_list += o
query = query.filter(
Employee.deleted == 0,
or_(and_(*and_list), *or_list)
)
return query
@staticmethod
def get_employee_list_by(department_id, block_status, search='', order='', page=1, page_size=10):
criterion = [
Employee.deleted == 0
]
if block_status >= 0:
criterion.append(
Employee.block == block_status
)
if len(search) > 0:
search_key = f"%{search}%"
criterion.append(
or_(
Employee.email.like(search_key),
Employee.username.like(search_key),
Employee.nickname.like(search_key)
)
)
if department_id > 0:
from api.lib.common_setting.department import DepartmentCRUD
department_id_list = DepartmentCRUD.get_department_id_list_by_root(
department_id)
criterion.append(
Employee.department_id.in_(department_id_list)
)
query = db.session.query(Employee, Department).outerjoin(Department).filter(
*criterion
)
if len(order) > 0:
query = EmployeeCRUD.format_query_sort(query, order)
pagination = query.paginate(page=page, per_page=page_size)
employees = []
for r in pagination.items:
d = r.Employee.to_dict()
d['department_name'] = r.Department.department_name
employees.append(d)
return {
'data_list': employees,
'page': page,
'page_size': page_size,
'total': pagination.total,
}
@staticmethod
def format_query_sort(query, order):
order_list = order.split(',')
all_columns = Employee.get_columns()
for order_column in order_list:
if order_column.startswith('-'):
target_column = order_column[1:]
if target_column not in all_columns:
continue
query = query.order_by(getattr(Employee, target_column).desc())
else:
if order_column not in all_columns:
continue
query = query.order_by(getattr(Employee, order_column).asc())
return query
@staticmethod
def get_employees_by_department_id(department_id, block):
criterion = [
Employee.deleted == 0,
Employee.block == block,
]
if type(department_id) == list:
if len(department_id) == 0:
return []
else:
criterion.append(
Employee.department_id.in_(department_id)
)
else:
criterion.append(
Employee.department_id == department_id
)
results = Employee.query.filter(
*criterion
).all()
return [r.to_dict() for r in results]
def get_user_map(key='uid', acl=None):
"""
{
uid: userinfo
}
"""
if acl is None:
acl = ACLManager()
data = {user[key]: user for user in acl.get_all_users()}
return data
acl_user_columns = [
'email',
'mobile',
'nickname',
'username',
'password',
'block',
'avatar',
]
employee_pop_columns = ['password']
can_not_edit_columns = ['email']
def format_params(params):
for k in ['_key', '_secret']:
params.pop(k, None)
return params
class CreateEmployee(object):
def __init__(self):
self.acl = ACLManager()
self.useremail_map = {}
def check_acl_user(self, email):
user_info = self.useremail_map.get(email, None)
if user_info:
return user_info
return None
def add_acl_user(self, **kwargs):
user_data = {column: kwargs.get(
column, '') for column in acl_user_columns if kwargs.get(column, '')}
try:
existed = self.check_acl_user(user_data['email'])
if not existed:
return self.acl.create_user(user_data)
return existed
except Exception as e:
abort(400, ErrFormat.acl_add_user_failed.format(str(e)))
def create_single(self, **kwargs):
EmployeeCRUD.check_email_unique(kwargs['email'])
self.useremail_map = self.useremail_map if self.useremail_map else get_user_map(
'email', self.acl)
user = self.add_acl_user(**kwargs)
kwargs['acl_uid'] = user['uid']
kwargs['last_login'] = user['last_login']
for column in employee_pop_columns:
kwargs.pop(column)
return Employee.create(
**kwargs
)
def create_single_with_import(self, **kwargs):
self.useremail_map = self.useremail_map if self.useremail_map else get_user_map(
'email', self.acl)
user = self.add_acl_user(**kwargs)
kwargs['acl_uid'] = user['uid']
kwargs['last_login'] = user['last_login']
for column in employee_pop_columns:
kwargs.pop(column)
existed = Employee.get_by(
first=True, to_dict=False, deleted=0, acl_uid=user['uid']
)
if existed:
return existed
return Employee.create(
**kwargs
)
def get_department_by_name(self, d_name):
return Department.get_by(first=True, department_name=d_name)
def get_end_department_id(self, department_name_list, department_name_map):
parent_id = 0
end_d_id = 0
for d_name in department_name_list:
tmp_d = self.get_department_by_name(d_name)
if not tmp_d:
tmp_d = Department.create(
department_name=d_name, department_parent_id=parent_id).to_dict()
else:
if tmp_d['department_parent_id'] != parent_id:
department_name_map[d_name] = tmp_d
raise Exception(ErrFormat.department_level_relation_error)
department_name_map[d_name] = tmp_d
end_d_id = tmp_d['department_id']
parent_id = tmp_d['department_id']
return end_d_id
def format_department_id(self, employee):
"""
部门名称转化为ID，不存在则创建
"""
department_name_map = {}
try:
department_name = employee.get('department_name', '')
if len(department_name) == 0:
return employee
department_name_list = department_name.split('/')
employee['department_id'] = self.get_end_department_id(
department_name_list, department_name_map)
except Exception as e:
employee['err'] = str(e)
return employee
def batch_create(self, employee_list):
err_list = []
self.useremail_map = get_user_map('email', self.acl)
for employee in employee_list:
try:
# 获取username
username = employee.get('username', None)
if username is None:
employee['username'] = employee['email']
# 校验通过后获取department_id
employee = self.format_department_id(employee)
err = employee.get('err', None)
if err:
raise Exception(err)
params = format_params(employee)
form = EmployeeAddForm(MultiDict(params))
if not form.validate():
raise Exception(
','.join(['{}: {}'.format(filed, ','.join(msg)) for filed, msg in form.errors.items()]))
data = self.create_single_with_import(**form.data)
except Exception as e:
err_list.append({
'email': employee.get('email', ''),
'nickname': employee.get('nickname', ''),
'err': str(e),
})
traceback.print_exc()
return err_list
class EmployeeAddForm(Form):
username = StringField(validators=[
validators.DataRequired(message="username不能为空"),
validators.Length(max=255),
])
email = StringField(validators=[
validators.DataRequired(message="邮箱不能为空"),
validators.Email(message="邮箱格式不正确"),
validators.Length(max=255),
])
password = StringField(validators=[
validators.Length(max=255),
])
position_name = StringField(validators=[])
nickname = StringField(validators=[
validators.DataRequired(message="用户名不能为空"),
validators.Length(max=255),
])
sex = StringField(validators=[])
mobile = StringField(validators=[])
department_id = IntegerField(validators=[], default=0)
direct_supervisor_id = IntegerField(validators=[], default=0)
class EmployeeUpdateByUidForm(Form):
nickname = StringField(validators=[
validators.DataRequired(message="用户名不能为空"),
validators.Length(max=255),
])
avatar = StringField(validators=[])
sex = StringField(validators=[])
mobile = StringField(validators=[])

51
cmdb-api/api/lib/common_setting/resp_format.py Normal file
View File

@ -0,0 +1,51 @@
# -*- coding:utf-8 -*-
from api.lib.resp_format import CommonErrFormat
class ErrFormat(CommonErrFormat):
company_info_is_already_existed = "公司信息已存在！无法创建"
no_file_part = "没有文件部分"
file_is_required = "文件是必须的"
direct_supervisor_is_not_self = "直属上级不能是自己"
parent_department_is_not_self = "上级部门不能是自己"
employee_list_is_empty = "员工列表为空"
column_name_not_support = "不支持的列名"
password_is_required = "密码不能为空"
employee_acl_rid_is_zero = "员工ACL角色ID不能为0"
generate_excel_failed = "生成excel失败: {}"
rename_columns_failed = "字段转换为中文失败: {}"
cannot_block_this_employee_is_other_direct_supervisor = "该员工是其他员工的直属上级, 不能禁用"
cannot_block_this_employee_is_department_manager = "该员工是部门负责人, 不能禁用"
employee_id_not_found = "员工ID [{}] 不存在"
value_is_required = "值是必须的"
email_already_exists = "邮箱 [{}] 已存在"
query_column_none_keep_value_empty = "查询 {} 空值时请保持value为空"
not_support_operator = "不支持的操作符: {}"
not_support_relation = "不支持的关系: {}"
conditions_field_missing = "conditions内元素字段缺失，请检查！"
datetime_format_error = "{} 格式错误，应该为：%Y-%m-%d %H:%M:%S"
department_level_relation_error = "部门层级关系不正确"
delete_reserved_department_name = "保留部门，无法删除！"
department_id_is_required = "部门ID是必须的"
department_list_is_required = "部门列表是必须的"
cannot_to_be_parent_department = "{} 不能设置为上级部门"
department_id_not_found = "部门ID [{}] 不存在"
parent_department_id_must_more_than_zero = "上级部门ID必须大于0"
department_name_already_exists = "部门名称 [{}] 已存在"
new_department_is_none = "新部门是空的"
acl_edit_user_failed = "ACL 修改用户失败: {}"
acl_uid_not_found = "ACL 用户UID [{}] 不存在"
acl_add_user_failed = "ACL 添加用户失败: {}"
acl_add_role_failed = "ACL 添加角色失败: {}"
acl_update_role_failed = "ACL 更新角色失败: {}"
acl_get_all_users_failed = "ACL 获取所有用户失败: {}"
acl_remove_user_from_role_failed = "ACL 从角色中移除用户失败: {}"
acl_add_user_to_role_failed = "ACL 添加用户到角色失败: {}"
acl_import_user_failed = "ACL 导入用户[{}]失败: {}"

16
cmdb-api/api/lib/common_setting/upload_file.py Normal file
View File

@ -0,0 +1,16 @@
import uuid
from api.lib.common_setting.utils import get_cur_time_str
def allowed_file(filename, allowed_extensions):
return '.' in filename and \
filename.rsplit('.', 1)[1].lower() in allowed_extensions
def generate_new_file_name(name):
ext = name.split('.')[-1]
prev_name = ''.join(name.split(f".{ext}")[:-1])
uid = str(uuid.uuid4())
cur_str = get_cur_time_str('_')
return f"{prev_name}_{cur_str}_{uid}.{ext}"

42
cmdb-api/api/lib/common_setting/utils.py Normal file
View File

@ -0,0 +1,42 @@
# -*- coding:utf-8 -*-
from datetime import datetime
import pandas as pd
from sqlalchemy import text
from api.extensions import db
def get_df_from_read_sql(query, to_dict=False):
bind = query.session.bind
query = query.statement.compile(dialect=bind.dialect if bind else None,
compile_kwargs={"literal_binds": True}).string
a = db.engine
df = pd.read_sql(sql=text(query), con=a.connect())
if to_dict:
return df.to_dict('records')
return df
def get_cur_time_str(split_flag='-'):
f = f"%Y{split_flag}%m{split_flag}%d{split_flag}%H{split_flag}%M{split_flag}%S{split_flag}%f"
return datetime.now().strftime(f)[:-3]
class BaseEnum(object):
_ALL_ = set()
@classmethod
def is_valid(cls, item):
return item in cls.all()
@classmethod
def all(cls):
if not cls._ALL_:
cls._ALL_ = {
getattr(cls, attr)
for attr in dir(cls)
if not attr.startswith("_") and not callable(getattr(cls, attr))
}
return cls._ALL_

17
cmdb-api/api/lib/const.py Normal file
View File

@ -0,0 +1,17 @@
# -*- coding:utf-8 -*-
from api.lib.utils import BaseEnum
class PermEnum(BaseEnum):
ADD = "create"
UPDATE = "update"
DELETE = "delete"
READ = "read"
EXECUTE = "execute"
GRANT = "grant"
ADMIN = "admin"
class RoleEnum(BaseEnum):
ADMIN = "OneOPS_Application_Admin"

95
cmdb-api/api/lib/database.py
View File

@ -10,36 +10,51 @@ from api.lib.exception import CommitException
class FormatMixin(object):
def to_dict(self):
res = dict()
for k in getattr(self, "__table__").columns:
if not isinstance(getattr(self, k.name), datetime.datetime):
res[k.name] = getattr(self, k.name)
else:
res[k.name] = getattr(self, k.name).strftime('%Y-%m-%d %H:%M:%S')
res = dict([(k, getattr(self, k) if not isinstance(
getattr(self, k), (datetime.datetime, datetime.date, datetime.time)) else str(
getattr(self, k))) for k in getattr(self, "__mapper__").c.keys()])
# FIXME: getattr(cls, "__table__").columns k.name
res.pop('password', None)
res.pop('_password', None)
res.pop('secret', None)
return res
@classmethod
def from_dict(cls, **kwargs):
from sqlalchemy.sql.sqltypes import Time, Date, DateTime
columns = dict(getattr(cls, "__table__").columns)
for k, c in columns.items():
if kwargs.get(k):
if type(c.type) == Time:
kwargs[k] = datetime.datetime.strptime(kwargs[k], "%H:%M:%S").time()
if type(c.type) == Date:
kwargs[k] = datetime.datetime.strptime(kwargs[k], "%Y-%m-%d").date()
if type(c.type) == DateTime:
kwargs[k] = datetime.datetime.strptime(kwargs[k], "%Y-%m-%d %H:%M:%S")
return cls(**kwargs)
@classmethod
def get_columns(cls):
return {k.name: 1 for k in getattr(cls, "__mapper__").c.values()}
return {k: 1 for k in getattr(cls, "__mapper__").c.keys()}
class CRUDMixin(FormatMixin):
def __init__(self, **kwargs):
super(CRUDMixin, self).__init__(**kwargs)
@classmethod
def create(cls, flush=False, **kwargs):
return cls(**kwargs).save(flush=flush)
def create(cls, flush=False, commit=True, **kwargs):
return cls(**kwargs).save(flush=flush, commit=commit)
def update(self, flush=False, **kwargs):
def update(self, flush=False, commit=True, filter_none=True, **kwargs):
kwargs.pop("id", None)
for attr, value in six.iteritems(kwargs):
if value is not None:
if (value is not None and filter_none) or not filter_none:
setattr(self, attr, value)
if flush:
return self.save(flush=flush)
return self.save()
return self.save(flush=flush, commit=commit)
def save(self, commit=True, flush=False):
db.session.add(self)
@ -54,12 +69,13 @@ class CRUDMixin(FormatMixin):
return self
def delete(self, flush=False):
def delete(self, flush=False, commit=True):
db.session.delete(self)
try:
if flush:
return db.session.flush()
return db.session.commit()
elif commit:
return db.session.commit()
except Exception as e:
db.session.rollback()
raise CommitException(str(e))
@ -73,10 +89,19 @@ class CRUDMixin(FormatMixin):
def get_by_id(cls, _id):
if any((isinstance(_id, six.string_types) and _id.isdigit(),
isinstance(_id, (six.integer_types, float))), ):
return getattr(cls, "query").get(int(_id)) or None
obj = getattr(cls, "query").get(int(_id))
if obj and not obj.deleted:
return obj
@classmethod
def get_by(cls, first=False, to_dict=True, fl=None, exclude=None, deleted=False, use_master=False, **kwargs):
def get_by(cls, first=False,
to_dict=True,
fl=None,
exclude=None,
deleted=False,
use_master=False,
only_query=False,
**kwargs):
db_session = db.session if not use_master else db.session().using_bind("master")
fl = fl.strip().split(",") if fl and isinstance(fl, six.string_types) else (fl or [])
exclude = exclude.strip().split(",") if exclude and isinstance(exclude, six.string_types) else (exclude or [])
@ -89,12 +114,26 @@ class CRUDMixin(FormatMixin):
if hasattr(cls, "deleted") and deleted is not None:
kwargs["deleted"] = deleted
kwargs_for_func = {i[7:]: kwargs[i] for i in kwargs if i.startswith('__func_')}
kwargs = {i: kwargs[i] for i in kwargs if not i.startswith('__func_')}
if fl:
query = db_session.query(*[getattr(cls, k) for k in fl])
query = query.filter_by(**kwargs)
result = [{k: getattr(i, k) for k in fl} for i in query]
else:
result = [i.to_dict() if to_dict else i for i in getattr(cls, 'query').filter_by(**kwargs)]
query = db_session.query(cls)
query = query.filter_by(**kwargs)
for i in kwargs_for_func:
func, key = i.split('__key_')
query = query.filter(getattr(getattr(cls, key), func)(kwargs_for_func[i]))
if only_query:
return query
if fl:
result = [{k: getattr(i, k) for k in fl} if to_dict else i for i in query]
else:
result = [i.to_dict() if to_dict else i for i in query]
return result[0] if first and result else (None if first else result)
@ -116,6 +155,10 @@ class TimestampMixin(object):
updated_at = db.Column(db.DateTime, onupdate=lambda: datetime.datetime.now())
class TimestampMixin2(object):
created_at = db.Column(db.DateTime, default=lambda: datetime.datetime.now(), index=True)
class SurrogatePK(object):
__table_args__ = {"extend_existing": True}
@ -128,3 +171,7 @@ class Model(SoftDeleteMixin, TimestampMixin, CRUDMixin, db.Model, SurrogatePK):
class CRUDModel(db.Model, CRUDMixin):
__abstract__ = True
class Model2(TimestampMixin2, db.Model, CRUDMixin, SurrogatePK):
__abstract__ = True

43
cmdb-api/api/lib/decorator.py
View File

@ -6,6 +6,8 @@ from functools import wraps
from flask import abort
from flask import request
from api.lib.resp_format import CommonErrFormat
def kwargs_required(*required_args):
def decorate(func):
@ -13,7 +15,8 @@ def kwargs_required(*required_args):
def wrapper(*args, **kwargs):
for arg in required_args:
if arg not in kwargs:
return abort(400, "Argument <{0}> is required".format(arg))
return abort(400, CommonErrFormat.argument_required.format(arg))
return func(*args, **kwargs)
return wrapper
@ -21,13 +24,47 @@ def kwargs_required(*required_args):
return decorate
def args_required(*required_args):
def args_required(*required_args, **value_required):
def decorate(func):
@wraps(func)
def wrapper(*args, **kwargs):
for arg in required_args:
if arg not in request.values:
return abort(400, "Argument <{0}> is required".format(arg))
return abort(400, CommonErrFormat.argument_required.format(arg))
if value_required.get('value_required', True) and not request.values.get(arg):
return abort(400, CommonErrFormat.argument_value_required.format(arg))
return func(*args, **kwargs)
return wrapper
return decorate
def args_validate(model_cls, exclude_args=None):
def decorate(func):
@wraps(func)
def wrapper(*args, **kwargs):
for arg in request.values:
if hasattr(model_cls, arg):
attr = getattr(model_cls, arg)
if not hasattr(attr, "type"):
continue
if exclude_args and arg in exclude_args:
continue
if attr.type.python_type == str and attr.type.length and \
len(request.values[arg] or '') > attr.type.length:
return abort(400, CommonErrFormat.argument_str_length_limit.format(arg, attr.type.length))
elif attr.type.python_type in (int, float) and request.values[arg]:
try:
int(float(request.values[arg]))
except (TypeError, ValueError):
return abort(400, CommonErrFormat.argument_invalid.format(arg))
return func(*args, **kwargs)
return wrapper

9
cmdb-api/api/lib/http_cli.py
View File

@ -1,18 +1,17 @@
# -*- coding:utf-8 -*-
from __future__ import unicode_literals
import hashlib
import requests
from flask import abort
from flask import current_app
from flask import g
from future.moves.urllib.parse import urlparse
from flask import abort
from flask import g
from flask import current_app
def build_api_key(path, params):
g.user is not None or abort(403, "You have to login to do this")
g.user is not None or abort(403, u"您得登陆才能进行该操作")
key = g.user.key
secret = g.user.secret
values = "".join([str(params[k]) for k in sorted(params.keys())

31
cmdb-api/api/lib/mail.py
View File

@ -1,51 +1,54 @@
# -*- coding:utf-8 -*-
import smtplib
import time
from email import Utils
from email.header import Header
from email.mime.image import MIMEImage
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.utils import make_msgid
from flask import current_app
def send_mail(sender, receiver, subject, content, ctype="html", pics=()):
"""subject and body are unicode objects"""
if not receiver:
return
if not sender:
sender = current_app.config.get("DEFAULT_MAIL_SENDER")
smtp_server = current_app.config.get("MAIL_SERVER")
smtpserver = current_app.config.get("MAIL_SERVER")
if ctype == "html":
msg = MIMEText(content, 'html', 'utf-8')
else:
msg = MIMEText(content, 'plain', 'utf-8')
if len(pics) != 0:
msg_root = MIMEMultipart('related')
msg_text = MIMEText(content, 'html', 'utf-8')
msg_root.attach(msg_text)
msgRoot = MIMEMultipart('related')
msgText = MIMEText(content, 'html', 'utf-8')
msgRoot.attach(msgText)
i = 1
for pic in pics:
fp = open(pic, "rb")
image = MIMEImage(fp.read())
fp.close()
image.add_header('Content-ID', '<img%02d>' % i)
msg_root.attach(image)
msgRoot.attach(image)
i += 1
msg = msg_root
msg = msgRoot
msg['Subject'] = Header(subject, 'utf-8')
msg['From'] = sender
msg['To'] = ';'.join(receiver)
msg['Message-ID'] = Utils.make_msgid()
msg['Message-ID'] = make_msgid()
msg['date'] = time.strftime('%a, %d %b %Y %H:%M:%S %z')
smtp = smtplib.SMTP()
smtp.connect(smtp_server, 25)
username, password = current_app.config.get("MAIL_USERNAME"), current_app.config.get("MAIL_PASSWORD")
if username and password:
smtp.login(username, password)
if current_app.config.get("MAIL_USE_SSL") or current_app.config.get("MAIL_USE_TLS"):
smtp = smtplib.SMTP_SSL(smtpserver)
else:
smtp = smtplib.SMTP()
smtp.connect(smtpserver, 25)
if current_app.config.get("MAIL_PASSWORD") != "":
smtp.login(current_app.config.get("MAIL_USERNAME"), current_app.config.get("MAIL_PASSWORD"))
smtp.sendmail(sender, receiver, msg.as_string())
smtp.quit()

96
cmdb-api/api/lib/mixin.py Normal file
View File

@ -0,0 +1,96 @@
# -*- coding:utf-8 -*-
from flask import abort
from sqlalchemy import func
from api.extensions import db
from api.lib.utils import get_page
from api.lib.utils import get_page_size
class DBMixin(object):
cls = None
@classmethod
def search(cls, page, page_size, fl=None, only_query=False, reverse=False, count_query=False, **kwargs):
page = get_page(page)
page_size = get_page_size(page_size)
if fl is None:
query = db.session.query(cls.cls).filter(cls.cls.deleted.is_(False))
else:
query = db.session.query(*[getattr(cls.cls, i) for i in fl]).filter(cls.cls.deleted.is_(False))
_query = None
if count_query:
_query = db.session.query(func.count(cls.cls.id)).filter(cls.cls.deleted.is_(False))
for k in kwargs:
if hasattr(cls.cls, k):
query = query.filter(getattr(cls.cls, k) == kwargs[k])
if count_query:
_query = _query.filter(getattr(cls.cls, k) == kwargs[k])
if reverse:
query = query.order_by(cls.cls.id.desc())
if only_query and not count_query:
return query
elif only_query:
return _query, query
numfound = query.count()
return numfound, [i.to_dict() if fl is None else getattr(i, '_asdict')()
for i in query.offset((page - 1) * page_size).limit(page_size)]
def _must_be_required(self, _id):
existed = self.cls.get_by_id(_id)
existed or abort(404, "Factor [{}] does not exist".format(_id))
return existed
def _can_add(self, **kwargs):
raise NotImplementedError
def _after_add(self, obj, **kwargs):
pass
def _can_update(self, **kwargs):
raise NotImplementedError
def _after_update(self, obj, **kwargs):
pass
def _can_delete(self, **kwargs):
raise NotImplementedError
def _after_delete(self, obj):
pass
def add(self, **kwargs):
kwargs = self._can_add(**kwargs) or kwargs
obj = self.cls.create(**kwargs)
kwargs['_id'] = obj.id if hasattr(obj, 'id') else None
self._after_add(obj, **kwargs)
return obj
def update(self, _id, **kwargs):
inst = self._can_update(_id=_id, **kwargs)
obj = inst.update(_id=_id, **kwargs)
self._after_update(obj, **kwargs)
return obj
def delete(self, _id):
inst = self._can_delete(_id=_id)
inst.soft_delete()
self._after_delete(inst)
return inst

16
cmdb-api/api/lib/perm/acl/__init__.py
View File

@ -3,20 +3,22 @@
from functools import wraps
from flask import request
from flask import abort
from flask import request
from api.lib.perm.acl.cache import AppCache
from api.lib.perm.acl.cache import AppCache, AppAccessTokenCache
from api.lib.perm.acl.resp_format import ErrFormat
def validate_app(func):
@wraps(func)
def wrapper(*args, **kwargs):
app_id = request.values.get('app_id')
app = AppCache.get(app_id)
if app is None:
return abort(400, "App <{0}> does not exist".format(app_id))
request.values['app_id'] = app.id
if not request.headers.get('App-Access-Token', '').strip():
app_id = request.values.get('app_id')
app = AppCache.get(app_id)
if app is None:
return abort(400, ErrFormat.app_not_found.format("id={}".format(app_id)))
request.values['app_id'] = app.id
return func(*args, **kwargs)

212
cmdb-api/api/lib/perm/acl/acl.py
View File

@ -1,37 +1,69 @@
# -*- coding:utf-8 -*-
import functools
import hashlib
import requests
import six
from flask import current_app, g, request
from flask import session, abort
from api.lib.cmdb.const import ResourceTypeEnum as CmdbResourceType
from api.lib.cmdb.const import RoleEnum
from api.extensions import cache
from api.lib.perm.acl.audit import AuditCRUD
from api.lib.perm.acl.cache import AppCache
from api.lib.perm.acl.cache import RoleCache
from api.lib.perm.acl.cache import UserCache
from api.lib.perm.acl.permission import PermissionCRUD
from api.lib.perm.acl.resource import ResourceCRUD
from api.lib.perm.acl.resp_format import ErrFormat
from api.lib.perm.acl.role import RoleCRUD
from api.lib.perm.acl.role import RoleRelationCRUD
from api.models.acl import App
from api.models.acl import Resource
from api.models.acl import ResourceGroup
from api.models.acl import ResourceType
from api.models.acl import Role
CMDB_RESOURCE_TYPES = CmdbResourceType.all()
def get_access_token():
url = "{0}/acl/apps/token".format(current_app.config.get('ACL_URI'))
payload = dict(app_id=current_app.config.get('APP_ID'),
secret_key=hashlib.md5(current_app.config.get('APP_SECRET_KEY').encode('utf-8')).hexdigest())
try:
res = requests.post(url, data=payload).json()
return res.get("token")
except Exception as e:
current_app.logger.error(str(e))
class AccessTokenCache(object):
TOKEN_KEY = 'TICKET::AccessToken'
@classmethod
def get(cls):
if cache.get(cls.TOKEN_KEY) is not None and cache.get(cls.TOKEN_KEY) != "":
return cache.get(cls.TOKEN_KEY)
res = get_access_token() or ""
cache.set(cls.TOKEN_KEY, res, timeout=60 * 60)
return res
@classmethod
def clean(cls):
cache.clear(cls.TOKEN_KEY)
class ACLManager(object):
def __init__(self):
self.app_id = AppCache.get('cmdb')
if not self.app_id:
raise Exception("cmdb not in acl apps")
self.app_id = self.app_id.id
def __init__(self, app=None):
self.app = AppCache.get(app or 'cmdb')
if not self.app:
raise Exception(ErrFormat.app_not_found.format(app))
self.app_id = self.app.id
def _get_resource(self, name, resource_type_name):
resource_type = ResourceType.get_by(name=resource_type_name, first=True, to_dict=False)
resource_type or abort(404, "ResourceType <{0}> cannot be found".format(resource_type_name))
resource_type or abort(404, ErrFormat.resource_type_not_found.format(resource_type_name))
return Resource.get_by(resource_type_id=resource_type.id,
app_id=self.app_id,
@ -52,13 +84,23 @@ class ACLManager(object):
if user:
return Role.get_by(name=name, uid=user.uid, first=True, to_dict=False)
return Role.get_by(name=name, app_id=self.app_id, first=True, to_dict=False)
return Role.get_by(name=name, app_id=self.app_id, first=True, to_dict=False) or \
Role.get_by(name=name, first=True, to_dict=False)
def add_resource(self, name, resource_type_name=None):
resource_type = ResourceType.get_by(name=resource_type_name, first=True, to_dict=False)
resource_type or abort(404, "ResourceType <{0}> cannot be found".format(resource_type_name))
resource_type or abort(404, ErrFormat.resource_type_not_found.format(resource_type_name))
ResourceCRUD.add(name, resource_type.id, self.app_id)
uid = AuditCRUD.get_current_operate_uid()
ResourceCRUD.add(name, resource_type.id, self.app_id, uid)
def update_resource(self, name, new_name, resource_type_name=None):
resource = self._get_resource(name, resource_type_name)
if resource is None:
self.add_resource(new_name, resource_type_name)
else:
ResourceCRUD.update(resource.id, new_name)
def grant_resource_to_role(self, name, role, resource_type_name=None, permissions=None):
resource = self._get_resource(name, resource_type_name)
@ -72,21 +114,103 @@ class ACLManager(object):
if group:
PermissionCRUD.grant(role.id, permissions, group_id=group.id)
def grant_resource_to_role_by_rid(self, name, rid, resource_type_name=None, permissions=None):
resource = self._get_resource(name, resource_type_name)
if resource:
PermissionCRUD.grant(rid, permissions, resource_id=resource.id)
else:
group = self._get_resource_group(name)
if group:
PermissionCRUD.grant(rid, permissions, group_id=group.id)
def revoke_resource_from_role(self, name, role, resource_type_name=None, permissions=None):
resource = self._get_resource(name, resource_type_name)
role = self._get_role(role)
if resource:
PermissionCRUD.revoke(role.id, permissions, resource_id=resource.id)
else:
group = self._get_resource_group(name)
if group:
PermissionCRUD.revoke(role.id, permissions, group_id=group.id)
def revoke_resource_from_role_by_rid(self, name, rid, resource_type_name=None, permissions=None):
resource = self._get_resource(name, resource_type_name)
if resource:
PermissionCRUD.revoke(rid, permissions, resource_id=resource.id)
else:
group = self._get_resource_group(name)
if group:
PermissionCRUD.revoke(rid, permissions, group_id=group.id)
def del_resource(self, name, resource_type_name=None):
resource = self._get_resource(name, resource_type_name)
if resource:
ResourceCRUD.delete(resource.id)
def has_permission(self, resource_name, resource_type, perm):
def has_permission(self, resource_name, resource_type, perm, resource_id=None):
if is_app_admin(self.app_id):
return True
role = self._get_role(g.user.username)
role or abort(404, "Role <{0}> is not found".format(g.user.username))
role or abort(404, ErrFormat.role_not_found.format(g.user.username))
return RoleCRUD.has_permission(role.id, resource_name, resource_type, self.app_id, perm)
return RoleCRUD.has_permission(role.id, resource_name, resource_type, self.app_id, perm,
resource_id=resource_id)
@staticmethod
def get_user_info(username, app_id=None):
user = UserCache.get(username)
if not user:
user = RoleCache.get_by_name(app_id, username) or RoleCache.get_by_name(None, username) # FIXME
if not user:
return abort(404, ErrFormat.user_not_found.format(username))
user = user.to_dict()
role = Role.get_by(uid=user['uid'], first=True, to_dict=False) if user.get('uid') else None
if role is not None:
user["rid"] = role.id
if app_id is None:
parent_ids = []
apps = App.get_by(to_dict=False)
for app in apps:
parent_ids.extend(RoleRelationCRUD.recursive_parent_ids(role.id, app.id))
else:
parent_ids = RoleRelationCRUD.recursive_parent_ids(role.id, app_id)
user['parents'] = [RoleCache.get(rid).name for rid in set(parent_ids) if RoleCache.get(rid)]
else:
user['parents'] = []
user['rid'] = user['id'] if user.get('id') else None
if user['rid']:
parent_ids = RoleRelationCRUD.recursive_parent_ids(user['rid'], app_id)
user['parents'] = [RoleCache.get(rid).name for rid in set(parent_ids) if RoleCache.get(rid)]
return user
def get_resources(self, resource_type_name=None):
role = self._get_role(g.user.username)
role or abort(404, ErrFormat.role_not_found.format(g.user.username))
rid = role.id
return RoleCRUD.recursive_resources(rid, self.app_id, resource_type_name).get('resources')
@staticmethod
def authenticate_with_token(token):
url = "{0}/acl/auth_with_token".format(current_app.config.get('ACL_URI'))
try:
return requests.post(url, json={"token": token},
headers={'App-Access-Token': AccessTokenCache.get()}).json()
except:
return {}
def validate_permission(resources, resource_type, perm):
def validate_permission(resources, resource_type, perm, app=None):
if not resources:
return
@ -96,11 +220,11 @@ def validate_permission(resources, resource_type, perm):
resources = [resources] if isinstance(resources, six.string_types) else resources
for resource in resources:
if not ACLManager().has_permission(resource, resource_type, perm):
return abort(403, "has no permission")
if not ACLManager(app).has_permission(resource, resource_type, perm):
return abort(403, ErrFormat.resource_no_permission.format(resource, perm))
def has_perm(resources, resource_type, perm):
def has_perm(resources, resource_type, perm, app=None):
def decorator_has_perm(func):
@functools.wraps(func)
def wrapper_has_perm(*args, **kwargs):
@ -108,10 +232,10 @@ def has_perm(resources, resource_type, perm):
return
if current_app.config.get("USE_ACL"):
if is_app_admin():
if is_app_admin(app):
return func(*args, **kwargs)
validate_permission(resources, resource_type, perm)
validate_permission(resources, resource_type, perm, app)
return func(*args, **kwargs)
@ -121,11 +245,14 @@ def has_perm(resources, resource_type, perm):
def is_app_admin(app=None):
if RoleEnum.CONFIG in session.get("acl", {}).get("parentRoles", []):
return True
app = app or 'cmdb'
app_id = AppCache.get(app).id
app = AppCache.get(app)
if app is None:
return False
app_id = app.id
if 'acl_admin' in session.get("acl", {}).get("parentRoles", []):
return True
for role_name in session.get("acl", {}).get("parentRoles", []):
role = RoleCache.get_by_name(app_id, role_name)
@ -135,7 +262,27 @@ def is_app_admin(app=None):
return False
def has_perm_from_args(arg_name, resource_type, perm, callback=None):
def is_admin():
if 'acl_admin' in session.get("acl", {}).get("parentRoles", []):
return True
return False
def admin_required(app=None):
def decorator_admin_required(func):
@functools.wraps(func)
def wrapper_admin_required(*args, **kwargs):
if is_app_admin(app):
return func(*args, **kwargs)
return abort(403, ErrFormat.admin_required)
return wrapper_admin_required
return decorator_admin_required
def has_perm_from_args(arg_name, resource_type, perm, callback=None, app=None):
def decorator_has_perm(func):
@functools.wraps(func)
def wrapper_has_perm(*args, **kwargs):
@ -146,10 +293,10 @@ def has_perm_from_args(arg_name, resource_type, perm, callback=None):
resource = callback(resource)
if current_app.config.get("USE_ACL") and resource:
if is_app_admin():
if is_app_admin(app):
return func(*args, **kwargs)
validate_permission(resource, resource_type, perm)
validate_permission(resource, resource_type, perm, app)
return func(*args, **kwargs)
@ -158,7 +305,7 @@ def has_perm_from_args(arg_name, resource_type, perm, callback=None):
return decorator_has_perm
def role_required(role_name):
def role_required(role_name, app=None):
def decorator_role_required(func):
@functools.wraps(func)
def wrapper_role_required(*args, **kwargs):
@ -166,8 +313,11 @@ def role_required(role_name):
return
if current_app.config.get("USE_ACL"):
if role_name not in session.get("acl", {}).get("parentRoles", []) and not is_app_admin():
return abort(403, "Role {0} is required".format(role_name))
if getattr(g.user, 'username', None) == "worker":
return func(*args, **kwargs)
if role_name not in session.get("acl", {}).get("parentRoles", []) and not is_app_admin(app):
return abort(403, ErrFormat.role_required.format(role_name))
return func(*args, **kwargs)
return wrapper_role_required

91
cmdb-api/api/lib/perm/acl/app.py Normal file
View File

@ -0,0 +1,91 @@
# -*- coding:utf-8 -*-
import datetime
import hashlib
import jwt
from flask import abort
from flask import current_app
from api.extensions import db
from api.lib.perm.acl.audit import AuditCRUD, AuditOperateType, AuditScope
from api.lib.perm.acl.resp_format import ErrFormat
from api.models.acl import App
class AppCRUD(object):
cls = App
@staticmethod
def get_all():
return App.get_by(to_dict=False)
@staticmethod
def get(app_id):
return App.get_by_id(app_id)
@staticmethod
def search(q, page=1, page_size=None):
query = db.session.query(App).filter(App.deleted.is_(False))
if q:
query = query.filter(App.name.ilike('%{0}%'.format(q)))
numfound = query.count()
res = query.offset((page - 1) * page_size).limit(page_size)
return numfound, res
@classmethod
def add(cls, name, description):
App.get_by(name=name) and abort(400, ErrFormat.app_is_ready_existed.format(name))
from api.lib.perm.acl.user import UserCRUD
app_id, secret_key = UserCRUD.gen_key_secret()
app = App.create(name=name, description=description, app_id=app_id, secret_key=secret_key)
AuditCRUD.add_resource_log(app.id, AuditOperateType.create, AuditScope.app, app.id, {}, app.to_dict(), {})
return app
@classmethod
def update(cls, _id, **kwargs):
kwargs.pop('id', None)
existed = App.get_by_id(_id) or abort(404, ErrFormat.app_not_found.format("id={}".format(_id)))
origin = existed.to_dict()
existed = existed.update(**kwargs)
AuditCRUD.add_resource_log(existed.id, AuditOperateType.update,
AuditScope.app, existed.id, origin, existed.to_dict(), {})
return existed
@classmethod
def delete(cls, _id):
app = App.get_by_id(_id) or abort(404, ErrFormat.app_not_found.format("id={}".format(_id)))
origin = app.to_dict()
app.soft_delete()
AuditCRUD.add_resource_log(app.id, AuditOperateType.delete,
AuditScope.app, app.id, origin, {}, {})
@staticmethod
def _get_by_key(key):
return App.get_by(app_id=key, first=True, to_dict=False)
@classmethod
def gen_token(cls, key, secret):
app = cls._get_by_key(key) or abort(404, ErrFormat.app_not_found.format("key={}".format(key)))
secret != hashlib.md5(app.secret_key.encode('utf-8')).hexdigest() and abort(403, ErrFormat.app_secret_invalid)
token = jwt.encode({
'sub': app.name,
'iat': datetime.datetime.now(),
'exp': datetime.datetime.now() + datetime.timedelta(minutes=2 * 60)},
current_app.config['SECRET_KEY'])
try:
return token.decode()
except AttributeError:
return token

355
cmdb-api/api/lib/perm/acl/audit.py Normal file
View File

@ -0,0 +1,355 @@
# -*- coding:utf-8 -*-
import itertools
import json
from enum import Enum
from typing import List
from flask import g, has_request_context, request
from flask_login import current_user
from sqlalchemy import func
from api.lib.perm.acl import AppCache
from api.models.acl import AuditRoleLog, AuditResourceLog, AuditPermissionLog, AuditTriggerLog, RolePermission, \
Resource, ResourceGroup, Permission, Role, ResourceType
class AuditScope(str, Enum):
app = 'app'
resource = 'resource'
resource_type = 'resource_type'
resource_group = 'resource_group'
user = 'user'
role = 'role'
role_relation = 'role_relation'
class AuditOperateType(str, Enum):
read = 'read'
create = 'create'
update = 'update'
delete = 'delete'
user_login = 'user_login'
role_relation_add = 'role_relation_add'
role_relation_delete = 'role_relation_delete'
grant = 'grant'
revoke = 'revoke'
trigger_apply = 'trigger_apply'
trigger_cancel = 'trigger_cancel'
class AuditOperateSource(str, Enum):
api = 'api'
acl = 'acl'
trigger = 'trigger'
class AuditCRUD(object):
@staticmethod
def get_current_operate_uid(uid=None):
user_id = uid or (hasattr(g, 'user') and getattr(g.user, 'uid', None)) \
or getattr(current_user, 'user_id', None)
if has_request_context() and request.headers.get('X-User-Id'):
_user_id = request.headers['X-User-Id']
user_id = int(_user_id) if _user_id.isdigit() else uid
return user_id
@staticmethod
def get_operate_source(source):
if has_request_context() and request.headers.get('App-Access-Token'):
source = AuditOperateSource.api
return source
@staticmethod
def search_permission(app_id, q=None, page=1, page_size=10, start=None, end=None):
criterion = []
if app_id:
app = AppCache.get(app_id)
criterion.append(AuditPermissionLog.app_id == app.id)
if start:
criterion.append(AuditPermissionLog.created_at >= start)
if end:
criterion.append(AuditPermissionLog.created_at <= end)
kwargs = {expr.split(':')[0]: expr.split(':')[1] for expr in q.split(',')} if q else {}
for k, v in kwargs.items():
if k == 'resource_type_id':
criterion.append(AuditPermissionLog.resource_type_id == int(v))
elif k == 'rid':
criterion.append(AuditPermissionLog.rid == int(v))
elif k == 'resource_id':
criterion.append(func.json_contains(AuditPermissionLog.resource_ids, v) == 1)
elif k == 'operate_uid':
criterion.append(AuditPermissionLog.operate_uid == v)
elif k == 'operate_type':
criterion.append(AuditPermissionLog.operate_type == v)
records = AuditPermissionLog.query.filter(
AuditPermissionLog.deleted == 0,
*criterion) \
.order_by(AuditPermissionLog.id.desc()) \
.offset((page - 1) * page_size) \
.limit(page_size).all()
data = {
'data': [r.to_dict() for r in records],
'id2resources': {},
'id2roles': {},
'id2groups': {},
'id2perms': {},
'id2resource_types': {},
}
resource_ids = set(itertools.chain(*[r.resource_ids for r in records]))
group_ids = set(itertools.chain(*[r.group_ids for r in records]))
permission_ids = set(itertools.chain(*[r.permission_ids for r in records]))
resource_type_ids = {r.resource_type_id for r in records}
rids = {r.rid for r in records}
if rids:
roles = Role.query.filter(Role.id.in_(rids)).all()
data['id2roles'] = {r.id: r.to_dict() for r in roles}
if resource_type_ids:
resource_types = ResourceType.query.filter(ResourceType.id.in_(resource_type_ids)).all()
data['id2resource_types'] = {r.id: r.to_dict() for r in resource_types}
if resource_ids:
resources = Resource.query.filter(Resource.id.in_(resource_ids)).all()
data['id2resources'] = {r.id: r.to_dict() for r in resources}
if group_ids:
groups = ResourceGroup.query.filter(ResourceGroup.id.in_(group_ids)).all()
data['id2groups'] = {_g.id: _g.to_dict() for _g in groups}
if permission_ids:
perms = Permission.query.filter(Permission.id.in_(permission_ids)).all()
data['id2perms'] = {_p.id: _p.to_dict() for _p in perms}
return data
@staticmethod
def search_role(app_id, q=None, page=1, page_size=10, start=None, end=None):
criterion = []
if app_id:
app = AppCache.get(app_id)
criterion.append(AuditRoleLog.app_id == app.id)
if start:
criterion.append(AuditRoleLog.created_at >= start)
if end:
criterion.append(AuditRoleLog.created_at <= end)
kwargs = {expr.split(':')[0]: expr.split(':')[1] for expr in q.split(',')} if q else {}
for k, v in kwargs.items():
if k == 'scope':
criterion.append(AuditRoleLog.scope == v)
elif k == 'link_id':
criterion.append(AuditRoleLog.link_id == int(v))
elif k == 'operate_uid':
criterion.append(AuditRoleLog.operate_uid == v)
elif k == 'operate_type':
criterion.append(AuditRoleLog.operate_type == v)
records = AuditRoleLog.query.filter(AuditRoleLog.deleted == 0, *criterion) \
.order_by(AuditRoleLog.id.desc()) \
.offset((page - 1) * page_size) \
.limit(page_size).all()
data = {
'data': [r.to_dict() for r in records],
'id2roles': {}
}
role_permissions = list(itertools.chain(*[r.extra.get('role_permissions', []) for r in records]))
_rids = set()
if role_permissions:
resource_ids = set([r['resource_id'] for r in role_permissions])
group_ids = set([r['group_id'] for r in role_permissions])
perm_ids = set([r['perm_id'] for r in role_permissions])
_rids.update(set([r['rid'] for r in role_permissions]))
if resource_ids:
resources = Resource.query.filter(Resource.id.in_(resource_ids)).all()
data['id2resources'] = {r.id: r.to_dict() for r in resources}
if group_ids:
groups = ResourceGroup.query.filter(ResourceGroup.id.in_(group_ids)).all()
data['id2groups'] = {_g.id: _g.to_dict() for _g in groups}
if perm_ids:
perms = Permission.query.filter(Permission.id.in_(perm_ids)).all()
data['id2perms'] = {_p.id: _p.to_dict() for _p in perms}
rids = set(itertools.chain(*[r.extra.get('child_ids', []) + r.extra.get('parent_ids', [])
for r in records]))
rids.update(_rids)
if rids:
roles = Role.query.filter(Role.id.in_(rids)).all()
data['id2roles'].update({r.id: r.to_dict() for r in roles})
return data
@staticmethod
def search_resource(app_id, q=None, page=1, page_size=10, start=None, end=None):
criterion = []
if app_id:
app = AppCache.get(app_id)
criterion.append(AuditResourceLog.app_id == app.id)
if start:
criterion.append(AuditResourceLog.created_at >= start)
if end:
criterion.append(AuditResourceLog.created_at <= end)
kwargs = {expr.split(':')[0]: expr.split(':')[1] for expr in q.split(',')} if q else {}
for k, v in kwargs.items():
if k == 'scope':
criterion.append(AuditResourceLog.scope == v)
elif k == 'link_id':
criterion.append(AuditResourceLog.link_id == int(v))
elif k == 'operate_uid':
criterion.append(AuditResourceLog.operate_uid == v)
elif k == 'operate_type':
criterion.append(AuditResourceLog.operate_type == v)
records = AuditResourceLog.query.filter(
AuditResourceLog.deleted == 0,
*criterion) \
.order_by(AuditResourceLog.id.desc()) \
.offset((page - 1) * page_size) \
.limit(page_size).all()
data = {
'data': [r.to_dict() for r in records],
}
return data
@staticmethod
def search_trigger(app_id, q=None, page=1, page_size=10, start=None, end=None):
criterion = []
if app_id:
app = AppCache.get(app_id)
criterion.append(AuditTriggerLog.app_id == app.id)
if start:
criterion.append(AuditTriggerLog.created_at >= start)
if end:
criterion.append(AuditTriggerLog.created_at <= end)
kwargs = {expr.split(':')[0]: expr.split(':')[1] for expr in q.split(',')} if q else {}
for k, v in kwargs.items():
if k == 'trigger_id':
criterion.append(AuditTriggerLog.trigger_id == int(v))
elif k == 'operate_uid':
criterion.append(AuditTriggerLog.operate_uid == v)
elif k == 'operate_type':
criterion.append(AuditTriggerLog.operate_type == v)
records = AuditTriggerLog.query.filter(
AuditTriggerLog.deleted == 0,
*criterion) \
.order_by(AuditTriggerLog.id.desc()) \
.offset((page - 1) * page_size) \
.limit(page_size).all()
data = {
'data': [r.to_dict() for r in records],
'id2roles': {},
'id2resource_types': {},
}
rids = set(itertools.chain(*[json.loads(r.origin.get('roles', "[]")) +
json.loads(r.current.get('roles', "[]"))
for r in records]))
resource_type_ids = set([r.origin.get('resource_type_id') for r in records
if r.origin.get('resource_type_id')] +
[r.current.get('resource_type_id') for r in records
if r.current.get('resource_type_id')])
if rids:
roles = Role.query.filter(Role.id.in_(rids)).all()
data['id2roles'] = {r.id: r.to_dict() for r in roles}
if resource_type_ids:
resource_types = ResourceType.query.filter(ResourceType.id.in_(resource_type_ids)).all()
data['id2resource_types'] = {r.id: r.to_dict() for r in resource_types}
return data
@classmethod
def add_role_log(cls, app_id, operate_type: AuditOperateType,
scope: AuditScope, link_id: int, origin: dict, current: dict, extra: dict,
uid=None, source=AuditOperateSource.acl):
user_id = cls.get_current_operate_uid(uid)
AuditRoleLog.create(app_id=app_id, operate_uid=user_id, operate_type=operate_type.value,
scope=scope.value,
link_id=link_id,
origin=origin,
current=current,
extra=extra,
source=source.value)
@classmethod
def add_resource_log(cls, app_id, operate_type: AuditOperateType,
scope: AuditScope, link_id: int, origin: dict, current: dict, extra: dict,
uid=None, source=AuditOperateSource.acl):
user_id = cls.get_current_operate_uid(uid)
source = cls.get_operate_source(source)
AuditResourceLog.create(app_id=app_id, operate_uid=user_id, operate_type=operate_type.value,
scope=scope.value,
link_id=link_id,
origin=origin,
current=current,
extra=extra,
source=source.value)
@classmethod
def add_permission_log(cls, app_id, operate_type: AuditOperateType,
rid: int, rt_id: int, role_permissions: List[RolePermission],
uid=None, source=AuditOperateSource.acl):
if not role_permissions:
return
user_id = cls.get_current_operate_uid(uid)
source = cls.get_operate_source(source)
resource_ids = list({r.resource_id for r in role_permissions if r.resource_id})
permission_ids = list({r.perm_id for r in role_permissions if r.perm_id})
group_ids = list({r.group_id for r in role_permissions if r.group_id})
AuditPermissionLog.create(app_id=app_id, operate_uid=user_id,
operate_type=operate_type.value,
rid=rid,
resource_type_id=rt_id,
resource_ids=resource_ids,
permission_ids=permission_ids,
group_ids=group_ids,
source=source.value)
@classmethod
def add_trigger_log(cls, app_id, trigger_id, operate_type: AuditOperateType,
origin: dict, current: dict, extra: dict,
uid=None, source=AuditOperateSource.acl):
user_id = cls.get_current_operate_uid(uid)
source = cls.get_operate_source(source)
AuditTriggerLog.create(app_id=app_id, trigger_id=trigger_id, operate_uid=user_id,
operate_type=operate_type.value,
origin=origin, current=current, extra=extra, source=source.value)

224
cmdb-api/api/lib/perm/acl/cache.py
View File

@ -1,13 +1,32 @@
# -*- coding:utf-8 -*-
import msgpack
from api.extensions import cache
from api.extensions import db
from api.lib.utils import Lock
from api.models.acl import App
from api.models.acl import Permission
from api.models.acl import Resource
from api.models.acl import ResourceGroup
from api.models.acl import Role
from api.models.acl import User
class AppAccessTokenCache(object):
PREFIX = "AppAccessTokenCache::token::{}"
@classmethod
def get_app_id(cls, token):
app_id = cache.get(cls.PREFIX.format(token))
return app_id
@classmethod
def set(cls, token, app, timeout=7200):
cache.set(token, cls.PREFIX.format(app.app_id), timeout=timeout)
class AppCache(object):
PREFIX_ID = "App::id::{0}"
PREFIX_NAME = "App::name::{0}"
@ -37,16 +56,19 @@ class UserCache(object):
PREFIX_ID = "User::uid::{0}"
PREFIX_NAME = "User::username::{0}"
PREFIX_NICK = "User::nickname::{0}"
PREFIX_WXID = "User::wxid::{0}"
@classmethod
def get(cls, key):
user = cache.get(cls.PREFIX_ID.format(key)) or \
cache.get(cls.PREFIX_NAME.format(key)) or \
cache.get(cls.PREFIX_NICK.format(key))
cache.get(cls.PREFIX_NICK.format(key)) or \
cache.get(cls.PREFIX_WXID.format(key))
if not user:
user = User.query.get(key) or \
User.query.get_by_username(key) or \
User.query.get_by_nickname(key)
User.query.get_by_nickname(key) or \
User.query.get_by_wxid(key)
if user:
cls.set(user)
@ -57,12 +79,16 @@ class UserCache(object):
cache.set(cls.PREFIX_ID.format(user.uid), user)
cache.set(cls.PREFIX_NAME.format(user.username), user)
cache.set(cls.PREFIX_NICK.format(user.nickname), user)
if user.wx_id:
cache.set(cls.PREFIX_WXID.format(user.wx_id), user)
@classmethod
def clean(cls, user):
cache.delete(cls.PREFIX_ID.format(user.uid))
cache.delete(cls.PREFIX_NAME.format(user.username))
cache.delete(cls.PREFIX_NICK.format(user.nickname))
if user.wx_id:
cache.delete(cls.PREFIX_WXID.format(user.wx_id))
class RoleCache(object):
@ -74,6 +100,9 @@ class RoleCache(object):
role = cache.get(cls.PREFIX_NAME.format(app_id, name))
if role is None:
role = Role.get_by(app_id=app_id, name=name, first=True, to_dict=False)
if role is None and app_id is None: # try global role
role = Role.get_by(name=name, first=True, to_dict=False)
if role is not None:
cache.set(cls.PREFIX_NAME.format(app_id, name), role)
@ -98,77 +127,196 @@ class RoleCache(object):
cache.delete(cls.PREFIX_NAME.format(app_id, name))
class RoleRelationCache(object):
PREFIX_PARENT = "RoleRelationParent::id::{0}"
PREFIX_CHILDREN = "RoleRelationChildren::id::{0}"
PREFIX_RESOURCES = "RoleRelationResources::id::{0}"
class HasResourceRoleCache(object):
PREFIX_KEY = "HasResourceRoleCache::AppId::{0}"
@classmethod
def get_parent_ids(cls, rid):
parent_ids = cache.get(cls.PREFIX_PARENT.format(rid))
def get(cls, app_id):
return cache.get(cls.PREFIX_KEY.format(app_id)) or {}
@classmethod
def add(cls, rid, app_id):
with Lock('HasResourceRoleCache'):
c = cls.get(app_id)
c[rid] = 1
cache.set(cls.PREFIX_KEY.format(app_id), c, timeout=0)
@classmethod
def remove(cls, rid, app_id):
with Lock('HasResourceRoleCache'):
c = cls.get(app_id)
c.pop(rid, None)
cache.set(cls.PREFIX_KEY.format(app_id), c, timeout=0)
class RoleRelationCache(object):
PREFIX_PARENT = "RoleRelationParent::id::{0}::AppId::{1}"
PREFIX_CHILDREN = "RoleRelationChildren::id::{0}::AppId::{1}"
PREFIX_RESOURCES = "RoleRelationResources::id::{0}::AppId::{1}"
PREFIX_RESOURCES2 = "RoleRelationResources2::id::{0}::AppId::{1}"
@classmethod
def get_parent_ids(cls, rid, app_id):
parent_ids = cache.get(cls.PREFIX_PARENT.format(rid, app_id))
if not parent_ids:
from api.lib.perm.acl.role import RoleRelationCRUD
parent_ids = RoleRelationCRUD.get_parent_ids(rid)
cache.set(cls.PREFIX_PARENT.format(rid), parent_ids, timeout=0)
parent_ids = RoleRelationCRUD.get_parent_ids(rid, app_id)
cache.set(cls.PREFIX_PARENT.format(rid, app_id), parent_ids, timeout=0)
return parent_ids
@classmethod
def get_child_ids(cls, rid):
child_ids = cache.get(cls.PREFIX_CHILDREN.format(rid))
def get_child_ids(cls, rid, app_id):
child_ids = cache.get(cls.PREFIX_CHILDREN.format(rid, app_id))
if not child_ids:
from api.lib.perm.acl.role import RoleRelationCRUD
child_ids = RoleRelationCRUD.get_child_ids(rid)
cache.set(cls.PREFIX_CHILDREN.format(rid), child_ids, timeout=0)
child_ids = RoleRelationCRUD.get_child_ids(rid, app_id)
cache.set(cls.PREFIX_CHILDREN.format(rid, app_id), child_ids, timeout=0)
return child_ids
@classmethod
def get_resources(cls, rid):
def get_resources(cls, rid, app_id):
"""
:param rid:
:param app_id:
:return: {id2perms: {resource_id: [perm,]}, group2perms: {group_id: [perm, ]}}
"""
resources = cache.get(cls.PREFIX_RESOURCES.format(rid))
resources = cache.get(cls.PREFIX_RESOURCES.format(rid, app_id))
if not resources:
from api.lib.perm.acl.role import RoleCRUD
resources = RoleCRUD.get_resources(rid)
cache.set(cls.PREFIX_RESOURCES.format(rid), resources, timeout=0)
resources = RoleCRUD.get_resources(rid, app_id)
if resources['id2perms'] or resources['group2perms']:
cache.set(cls.PREFIX_RESOURCES.format(rid, app_id), resources, timeout=0)
return resources or {}
@classmethod
def rebuild(cls, rid):
cls.clean(rid)
db.session.close()
cls.get_parent_ids(rid)
cls.get_child_ids(rid)
cls.get_resources(rid)
def get_resources2(cls, rid, app_id):
r_g = cache.get(cls.PREFIX_RESOURCES2.format(rid, app_id))
if not r_g:
res = cls.get_resources(rid, app_id)
id2perms = res['id2perms']
group2perms = res['group2perms']
resources, groups = dict(), dict()
for _id in id2perms:
resource = ResourceCache.get(_id)
if not resource:
continue
resource = resource.to_dict()
resource.update(dict(permissions=id2perms[_id]))
resources[_id] = resource
for _id in group2perms:
group = ResourceGroupCache.get(_id)
if not group:
continue
group = group.to_dict()
group.update(dict(permissions=group2perms[_id]))
groups[_id] = group
r_g = msgpack.dumps(dict(resources=resources, groups=groups))
cache.set(cls.PREFIX_RESOURCES2.format(rid, app_id), r_g, timeout=0)
return msgpack.loads(r_g, raw=False)
@classmethod
def clean(cls, rid):
cache.delete(cls.PREFIX_PARENT.format(rid))
cache.delete(cls.PREFIX_CHILDREN.format(rid))
cache.delete(cls.PREFIX_RESOURCES.format(rid))
def rebuild(cls, rid, app_id):
cls.clean(rid, app_id)
db.session.remove()
cls.get_parent_ids(rid, app_id)
cls.get_child_ids(rid, app_id)
resources = cls.get_resources(rid, app_id)
if resources.get('id2perms') or resources.get('group2perms'):
HasResourceRoleCache.add(rid, app_id)
else:
HasResourceRoleCache.remove(rid, app_id)
cls.get_resources2(rid, app_id)
@classmethod
def rebuild2(cls, rid, app_id):
cache.delete(cls.PREFIX_RESOURCES2.format(rid, app_id))
db.session.remove()
cls.get_resources2(rid, app_id)
@classmethod
def clean(cls, rid, app_id):
cache.delete(cls.PREFIX_PARENT.format(rid, app_id))
cache.delete(cls.PREFIX_CHILDREN.format(rid, app_id))
cache.delete(cls.PREFIX_RESOURCES.format(rid, app_id))
cache.delete(cls.PREFIX_RESOURCES2.format(rid, app_id))
class PermissionCache(object):
PREFIX_ID = "Permission::id::{0}"
PREFIX_NAME = "Permission::name::{0}"
PREFIX_ID = "Permission::id::{0}::ResourceTypeId::{1}"
PREFIX_NAME = "Permission::name::{0}::ResourceTypeId::{1}"
@classmethod
def get(cls, key):
perm = cache.get(cls.PREFIX_ID.format(key))
perm = perm or cache.get(cls.PREFIX_NAME.format(key))
def get(cls, key, rt_id):
perm = cache.get(cls.PREFIX_ID.format(key, rt_id))
perm = perm or cache.get(cls.PREFIX_NAME.format(key, rt_id))
if perm is None:
perm = Permission.get_by_id(key)
perm = perm or Permission.get_by(name=key, first=True, to_dict=False)
perm = perm or Permission.get_by(name=key, resource_type_id=rt_id, first=True, to_dict=False)
if perm is not None:
cache.set(cls.PREFIX_ID.format(key), perm)
cache.set(cls.PREFIX_ID.format(perm.id, rt_id), perm)
cache.set(cls.PREFIX_NAME.format(perm.name, rt_id), perm)
return perm
class ResourceCache(object):
PREFIX_ID = "Resource::id::{0}"
PREFIX_NAME = "Resource::type_id::{0}::name::{1}"
@classmethod
def clean(cls, key):
cache.delete(cls.PREFIX_ID.format(key))
cache.delete(cls.PREFIX_NAME.format(key))
def get(cls, key, type_id=None):
resource = cache.get(cls.PREFIX_ID.format(key)) or cache.get(cls.PREFIX_NAME.format(type_id, key))
if resource is None:
resource = Resource.get_by_id(key) or Resource.get_by(name=key,
resource_type_id=type_id,
to_dict=False,
first=True)
if resource is not None:
cls.set(resource)
return resource
@classmethod
def set(cls, resource):
cache.set(cls.PREFIX_ID.format(resource.id), resource)
cache.set(cls.PREFIX_NAME.format(resource.resource_type_id, resource.name), resource)
@classmethod
def clean(cls, resource):
cache.delete(cls.PREFIX_ID.format(resource.id))
cache.delete(cls.PREFIX_NAME.format(resource.resource_type_id, resource.name))
class ResourceGroupCache(object):
PREFIX_ID = "ResourceGroup::id::{0}"
PREFIX_NAME = "ResourceGroup::type_id::{0}::name::{1}"
@classmethod
def get(cls, key, type_id=None):
group = cache.get(cls.PREFIX_ID.format(key)) or cache.get(cls.PREFIX_NAME.format(type_id, key))
if group is None:
group = ResourceGroup.get_by_id(key) or ResourceGroup.get_by(name=key,
resource_type_id=type_id,
to_dict=False,
first=True)
if group is not None:
cls.set(group)
return group
@classmethod
def set(cls, group):
cache.set(cls.PREFIX_ID.format(group.id), group)
cache.set(cls.PREFIX_NAME.format(group.resource_type_id, group.name), group)
@classmethod
def clean(cls, group):
cache.delete(cls.PREFIX_ID.format(group.id))
cache.delete(cls.PREFIX_NAME.format(group.resource_type_id, group.name))

12
cmdb-api/api/lib/perm/acl/const.py
View File

@ -1,5 +1,13 @@
# -*- coding:utf-8 -*-
from api.lib.cmdb.const import CMDB_QUEUE
from api.lib.utils import BaseEnum
ACL_QUEUE = CMDB_QUEUE
ACL_QUEUE = "acl_async"
class OperateType(BaseEnum):
LOGIN = "0"
READ = "1"
UPDATE = "2"
CREATE = "3"
DELETE = "4"

282
cmdb-api/api/lib/perm/acl/permission.py
View File

@ -1,48 +1,302 @@
# -*- coding:utf-8 -*-
import datetime
from flask import abort
from api.extensions import db
from api.lib.perm.acl.audit import AuditCRUD, AuditOperateType, AuditOperateSource
from api.lib.perm.acl.cache import PermissionCache
from api.lib.perm.acl.cache import RoleCache
from api.lib.perm.acl.cache import UserCache
from api.lib.perm.acl.const import ACL_QUEUE
from api.lib.perm.acl.resp_format import ErrFormat
from api.lib.perm.acl.role import RoleRelationCRUD
from api.models.acl import Resource
from api.models.acl import ResourceGroup
from api.models.acl import ResourceType
from api.models.acl import RolePermission
from api.tasks.acl import role_rebuild
class PermissionCRUD(object):
@staticmethod
def get_all(resource_id=None, group_id=None):
def get_all(resource_id=None, group_id=None, need_users=True):
result = dict()
if resource_id is not None:
r = Resource.get_by_id(resource_id)
if not r:
return result
rt_id = r.resource_type_id
perms = RolePermission.get_by(resource_id=resource_id, to_dict=False)
else:
rg = ResourceGroup.get_by_id(group_id)
if not rg:
return result
rt_id = rg.resource_type_id
perms = RolePermission.get_by(group_id=group_id, to_dict=False)
rid2obj = dict()
uid2obj = dict()
for perm in perms:
perm_dict = PermissionCache.get(perm.perm_id).to_dict()
perm_dict = PermissionCache.get(perm.perm_id, rt_id)
perm_dict = perm_dict and perm_dict.to_dict()
if not perm_dict:
continue
perm_dict.update(dict(rid=perm.rid))
result.setdefault(RoleCache.get(perm.rid).name, []).append(perm_dict)
if perm.rid not in rid2obj:
rid2obj[perm.rid] = RoleCache.get(perm.rid)
role = rid2obj[perm.rid]
if role and role.uid:
if role.uid not in uid2obj:
uid2obj[role.uid] = UserCache.get(role.uid)
name = uid2obj[role.uid].nickname
elif role:
name = role.name
else:
continue
result.setdefault(name,
dict(perms=[],
users=RoleRelationCRUD.get_users_by_rid(perm.rid, perm.app_id, rid2obj, uid2obj)
if need_users else [])
)['perms'].append(perm_dict)
return result
@staticmethod
def grant(rid, perms, resource_id=None, group_id=None):
for perm in perms:
perm = PermissionCache.get(perm)
existed = RolePermission.get_by(rid=rid, perm_id=perm.id, group_id=group_id, resource_id=resource_id)
existed or RolePermission.create(rid=rid, perm_id=perm.id, group_id=group_id, resource_id=resource_id)
@classmethod
def get_all2(cls, resource_name, resource_type_name, app_id):
rt = ResourceType.get_by(name=resource_type_name, first=True, to_dict=False)
rt or abort(404, ErrFormat.resource_type_not_found.format(resource_type_name))
role_rebuild.apply_async(args=(rid,), queue=ACL_QUEUE)
r = Resource.get_by(name=resource_name, resource_type_id=rt.id, app_id=app_id, first=True, to_dict=False)
return r and cls.get_all(r.id)
@staticmethod
def revoke(rid, perms, resource_id=None, group_id=None):
def grant(rid, perms, resource_id=None, group_id=None, rebuild=True, source=AuditOperateSource.acl):
app_id = None
rt_id = None
from api.lib.perm.acl.resource import ResourceTypeCRUD
if resource_id is not None:
from api.models.acl import Resource
resource = Resource.get_by_id(resource_id) or abort(404, ErrFormat.resource_not_found.format(
"id={}".format(resource_id)))
app_id = resource.app_id
rt_id = resource.resource_type_id
if not perms:
perms = [i.get('name') for i in ResourceTypeCRUD.get_perms(resource.resource_type_id)]
elif group_id is not None:
from api.models.acl import ResourceGroup
group = ResourceGroup.get_by_id(group_id) or \
abort(404, ErrFormat.resource_group_not_found.format("id={}".format(group_id)))
app_id = group.app_id
rt_id = group.resource_type_id
if not perms:
perms = [i.get('name') for i in ResourceTypeCRUD.get_perms(group.resource_type_id)]
_role_permissions = []
for _perm in set(perms):
perm = PermissionCache.get(_perm, rt_id)
if not perm:
continue
existed = RolePermission.get_by(rid=rid,
app_id=app_id,
perm_id=perm.id,
group_id=group_id,
resource_id=resource_id)
if not existed:
__role_permission = RolePermission.create(rid=rid,
app_id=app_id,
perm_id=perm.id,
group_id=group_id,
resource_id=resource_id)
_role_permissions.append(__role_permission)
if rebuild:
role_rebuild.apply_async(args=(rid, app_id), queue=ACL_QUEUE)
AuditCRUD.add_permission_log(app_id, AuditOperateType.grant, rid, rt_id, _role_permissions,
source=source)
@staticmethod
def batch_grant_by_resource_names(rid, perms, resource_type_id, resource_names,
resource_ids=None, perm_map=None, app_id=None):
from api.lib.perm.acl.resource import ResourceTypeCRUD
if resource_names:
resource_ids = []
from api.models.acl import Resource
for n in resource_names:
resource = Resource.get_by(name=n, resource_type_id=resource_type_id, first=True, to_dict=False)
if resource:
app_id = resource.app_id
if not perms:
perms = [i.get('name') for i in ResourceTypeCRUD.get_perms(resource.resource_type_id)]
resource_ids.append(resource.id)
resource_ids = resource_ids or []
_role_permissions = []
if isinstance(perm_map, dict):
perm2resource = dict()
for resource_id in resource_ids:
for _perm in (perm_map.get(str(resource_id)) or []):
perm2resource.setdefault(_perm, []).append(resource_id)
for _perm in perm2resource:
perm = PermissionCache.get(_perm, resource_type_id)
existeds = RolePermission.get_by(rid=rid,
app_id=app_id,
perm_id=perm.id,
__func_in___key_resource_id=perm2resource[_perm],
to_dict=False)
for resource_id in (set(perm2resource[_perm]) - set([i.resource_id for i in existeds])):
_role_permission = RolePermission.create(flush=False,
commit=False,
rid=rid,
app_id=app_id,
perm_id=perm.id,
resource_id=resource_id,
)
_role_permissions.append(_role_permission)
db.session.commit()
else:
for _perm in perms:
perm = PermissionCache.get(_perm, resource_type_id)
for resource_id in resource_ids:
existed = RolePermission.get_by(rid=rid,
app_id=app_id,
perm_id=perm.id,
resource_id=resource_id)
if not existed:
_role_permission = RolePermission.create(rid=rid,
app_id=app_id,
perm_id=perm.id,
resource_id=resource_id)
_role_permissions.append(_role_permission)
role_rebuild.apply_async(args=(rid, app_id), queue=ACL_QUEUE)
AuditCRUD.add_permission_log(app_id, AuditOperateType.grant, rid, resource_type_id, _role_permissions)
@staticmethod
def revoke(rid, perms, resource_id=None, group_id=None, rebuild=True, source=AuditOperateSource.acl):
app_id = None
rt_id = None
from api.lib.perm.acl.resource import ResourceTypeCRUD
if resource_id is not None:
from api.models.acl import Resource
resource = Resource.get_by_id(resource_id) or \
abort(404, ErrFormat.resource_not_found.format("id={}".format(resource_id)))
app_id = resource.app_id
rt_id = resource.resource_type_id
if not perms:
perms = [i.get('name') for i in ResourceTypeCRUD.get_perms(resource.resource_type_id)]
elif group_id is not None:
from api.models.acl import ResourceGroup
group = ResourceGroup.get_by_id(group_id) or \
abort(404, ErrFormat.resource_group_not_found.format("id={}".format(group_id)))
app_id = group.app_id
rt_id = group.resource_type_id
if not perms:
perms = [i.get('name') for i in ResourceTypeCRUD.get_perms(group.resource_type_id)]
_role_permissions = []
for perm in perms:
perm = PermissionCache.get(perm)
perm = PermissionCache.get(perm, rt_id)
if not perm:
continue
existed = RolePermission.get_by(rid=rid,
perm_id=perm.id,
group_id=group_id,
resource_id=resource_id,
first=True,
to_dict=False)
existed and existed.soft_delete()
if existed:
existed.soft_delete()
_role_permissions.append(existed)
role_rebuild.apply_async(args=(rid,), queue=ACL_QUEUE)
if rebuild:
role_rebuild.apply_async(args=(rid, app_id), queue=ACL_QUEUE)
AuditCRUD.add_permission_log(app_id, AuditOperateType.revoke, rid, rt_id, _role_permissions,
source=source)
@staticmethod
def batch_revoke_by_resource_names(rid, perms, resource_type_id, resource_names,
resource_ids=None, perm_map=None, app_id=None):
from api.lib.perm.acl.resource import ResourceTypeCRUD
if resource_names:
resource_ids = []
from api.models.acl import Resource
for n in resource_names:
resource = Resource.get_by(name=n, resource_type_id=resource_type_id, first=True, to_dict=False)
if resource:
app_id = resource.app_id
if not perms:
perms = [i.get('name') for i in ResourceTypeCRUD.get_perms(resource.resource_type_id)]
resource_ids.append(resource.id)
resource_ids = resource_ids or []
_role_permissions = []
if isinstance(perm_map, dict):
perm2resource = dict()
for resource_id in resource_ids:
for _perm in (perm_map.get(str(resource_id)) or []):
perm2resource.setdefault(_perm, []).append(resource_id)
for _perm in perm2resource:
perm = PermissionCache.get(_perm, resource_type_id)
existeds = RolePermission.get_by(rid=rid,
app_id=app_id,
perm_id=perm.id,
__func_in___key_resource_id=perm2resource[_perm],
to_dict=False)
for existed in existeds:
existed.deleted = True
existed.deleted_at = datetime.datetime.now()
db.session.add(existed)
_role_permissions.append(existed)
db.session.commit()
else:
for _perm in perms:
perm = PermissionCache.get(_perm, resource_type_id)
for resource_id in resource_ids:
existed = RolePermission.get_by(rid=rid,
app_id=app_id,
perm_id=perm.id,
resource_id=resource_id,
first=True, to_dict=False)
if existed:
existed.soft_delete()
_role_permissions.append(existed)
role_rebuild.apply_async(args=(rid, app_id), queue=ACL_QUEUE)
AuditCRUD.add_permission_log(app_id, AuditOperateType.revoke, rid, resource_type_id, _role_permissions)

21
cmdb-api/api/lib/perm/acl/record.py Normal file
View File

@ -0,0 +1,21 @@
# -*- coding:utf-8 -*-
from api.models.acl import OperationRecord
class OperateRecordCRUD(object):
@staticmethod
def search(page, page_size, **kwargs):
query = OperationRecord.get_by(only_query=True)
for k, v in kwargs.items():
if hasattr(OperationRecord, k) and v:
query = query.filter(getattr(OperationRecord, k) == v)
numfound = query.count()
res = query.offset((page - 1) * page_size).limit(page_size)
return numfound, res
@staticmethod
def add(app, rolename, operate, obj):
OperationRecord.create(app=app, rolename=rolename, operate=operate, obj=obj)

224
cmdb-api/api/lib/perm/acl/resource.py
View File

@ -2,9 +2,16 @@
from flask import abort
from flask import current_app
from api.extensions import db
from api.lib.perm.acl.audit import AuditCRUD, AuditOperateType, AuditScope
from api.lib.perm.acl.cache import ResourceCache
from api.lib.perm.acl.cache import ResourceGroupCache
from api.lib.perm.acl.cache import UserCache
from api.lib.perm.acl.const import ACL_QUEUE
from api.lib.perm.acl.resp_format import ErrFormat
from api.lib.perm.acl.trigger import TriggerCRUD
from api.models.acl import Permission
from api.models.acl import Resource
from api.models.acl import ResourceGroup
@ -12,9 +19,12 @@ from api.models.acl import ResourceGroupItems
from api.models.acl import ResourceType
from api.models.acl import RolePermission
from api.tasks.acl import role_rebuild
from api.tasks.acl import update_resource_to_build_role
class ResourceTypeCRUD(object):
cls = ResourceType
@staticmethod
def search(q, app_id, page=1, page_size=None):
query = db.session.query(ResourceType).filter(
@ -33,6 +43,15 @@ class ResourceTypeCRUD(object):
return numfound, res, id2perms
@classmethod
def id2name(cls):
return {i.id: i.name for i in ResourceType.get_by(to_dict=False)}
@staticmethod
def get_by_name(app_id, name):
resource_type = ResourceType.get_by(first=True, app_id=app_id, name=name, to_dict=False)
return resource_type
@staticmethod
def get_perms(rt_id):
perms = Permission.get_by(resource_type_id=rt_id, to_dict=False)
@ -40,12 +59,16 @@ class ResourceTypeCRUD(object):
@classmethod
def add(cls, app_id, name, description, perms):
ResourceType.get_by(name=name, app_id=app_id) and abort(
400, "ResourceType <{0}> is already existed".format(name))
ResourceType.get_by(name=name, app_id=app_id) and abort(400, ErrFormat.resource_type_exists.format(name))
rt = ResourceType.create(name=name, description=description, app_id=app_id)
cls.update_perms(rt.id, perms, app_id)
_, current_perm_ids = cls.update_perms(rt.id, perms, app_id)
AuditCRUD.add_resource_log(app_id, AuditOperateType.create,
AuditScope.resource_type, rt.id, {}, rt.to_dict(),
{'permission_ids': {'current': current_perm_ids, 'origin': []}, }
)
return rt
@ -53,49 +76,79 @@ class ResourceTypeCRUD(object):
def update(cls, rt_id, **kwargs):
kwargs.pop('app_id', None)
rt = ResourceType.get_by_id(rt_id) or abort(404, "ResourceType <{0}> is not found".format(rt_id))
rt = ResourceType.get_by_id(rt_id) or abort(404,
ErrFormat.resource_type_not_found.format("id={}".format(rt_id)))
if 'name' in kwargs:
other = ResourceType.get_by(name=kwargs['name'], app_id=rt.app_id, to_dict=False, first=True)
if other and other.id != rt_id:
return abort(400, "ResourceType <{0}> is duplicated".format(kwargs['name']))
return abort(400, ErrFormat.resource_type_exists.format(kwargs['name']))
if 'perms' in kwargs:
cls.update_perms(rt_id, kwargs.pop('perms'), rt.app_id)
perms = kwargs.pop('perms', None)
current_perm_ids = []
existed_perm_ids = []
return rt.update(**kwargs)
if perms:
existed_perm_ids, current_perm_ids = cls.update_perms(rt_id, perms, rt.app_id)
origin = rt.to_dict()
rt = rt.update(**kwargs)
AuditCRUD.add_resource_log(rt.app_id, AuditOperateType.update,
AuditScope.resource_type, rt.id, origin, rt.to_dict(),
{'permission_ids': {'current': current_perm_ids, 'origin': existed_perm_ids}, }
)
return rt
@classmethod
def delete(cls, rt_id):
rt = ResourceType.get_by_id(rt_id) or abort(404, "ResourceType <{0}> is not found".format(rt_id))
rt = ResourceType.get_by_id(rt_id) or \
abort(404, ErrFormat.resource_type_not_found.format("id={}".format(rt_id)))
if Resource.get_by(resource_type_id=rt_id):
return abort(400, "At least one instance of this type exists and cannot be deleted")
Resource.get_by(resource_type_id=rt_id) and abort(400, ErrFormat.resource_type_cannot_delete)
cls.update_perms(rt_id, [], rt.app_id)
origin = rt.to_dict()
existed_perm_ids, _ = cls.update_perms(rt_id, [], rt.app_id)
rt.soft_delete()
AuditCRUD.add_resource_log(rt.app_id, AuditOperateType.delete,
AuditScope.resource_type, rt.id, origin, {},
{'permission_ids': {'current': [], 'origin': existed_perm_ids}, }
)
@classmethod
def update_perms(cls, rt_id, perms, app_id):
existed = Permission.get_by(resource_type_id=rt_id, to_dict=False)
existed_names = [i.name for i in existed]
existed_ids = [i.id for i in existed]
current_ids = []
for i in existed:
if i.name not in perms:
i.soft_delete()
else:
current_ids.append(i.id)
for i in perms:
if i not in existed_names:
Permission.create(resource_type_id=rt_id,
name=i,
app_id=app_id)
p = Permission.create(resource_type_id=rt_id,
name=i,
app_id=app_id)
current_ids.append(p.id)
return existed_ids, current_ids
class ResourceGroupCRUD(object):
cls = ResourceGroup
@staticmethod
def search(q, app_id, page=1, page_size=None):
def search(q, app_id, resource_type_id, page=1, page_size=None):
query = db.session.query(ResourceGroup).filter(
ResourceGroup.deleted.is_(False)).filter(ResourceGroup.app_id == app_id)
ResourceGroup.deleted.is_(False)).filter(ResourceGroup.app_id == app_id).filter(
ResourceGroup.resource_type_id == resource_type_id)
if q:
query = query.filter(ResourceGroup.name.ilike("%{0}%".format(q)))
@ -111,14 +164,20 @@ class ResourceGroupCRUD(object):
return [i.resource.to_dict() for i in items]
@staticmethod
def add(name, type_id, app_id):
ResourceGroup.get_by(name=name, resource_type_id=type_id, app_id=app_id) and abort(
400, "ResourceGroup <{0}> is already existed".format(name))
def add(name, type_id, app_id, uid=None):
ResourceGroup.get_by(name=name, resource_type_id=type_id, app_id=app_id) and \
abort(400, ErrFormat.resource_group_exists.format(name))
rg = ResourceGroup.create(name=name, resource_type_id=type_id, app_id=app_id, uid=uid)
return ResourceGroup.create(name=name, resource_type_id=type_id, app_id=app_id)
AuditCRUD.add_resource_log(app_id, AuditOperateType.create,
AuditScope.resource_group, rg.id, {}, rg.to_dict(), {})
return rg
@staticmethod
def update(rg_id, items):
rg = ResourceGroup.get_by_id(rg_id) or \
abort(404, ErrFormat.resource_group_not_found.format("id={}".format(rg_id)))
existed = ResourceGroupItems.get_by(group_id=rg_id, to_dict=False)
existed_ids = [i.resource_id for i in existed]
@ -130,60 +189,153 @@ class ResourceGroupCRUD(object):
if _id not in existed_ids:
ResourceGroupItems.create(group_id=rg_id, resource_id=_id)
AuditCRUD.add_resource_log(rg.app_id, AuditOperateType.update,
AuditScope.resource_group, rg.id, rg.to_dict(), rg.to_dict(),
{'resource_ids': {'current': items, 'origin': existed_ids}, }
)
@staticmethod
def delete(rg_id):
rg = ResourceGroup.get_by_id(rg_id) or abort(404, "ResourceGroup <{0}> is not found".format(rg_id))
rg = ResourceGroup.get_by_id(rg_id) or \
abort(404, ErrFormat.resource_group_not_found.format("id={}".format(rg_id)))
origin = rg.to_dict()
rg.soft_delete()
items = ResourceGroupItems.get_by(group_id=rg_id, to_dict=False)
existed_ids = []
for item in items:
existed_ids.append(item.resource_id)
item.soft_delete()
rebuild = set()
for i in RolePermission.get_by(group_id=rg_id, to_dict=False):
i.soft_delete()
role_rebuild.apply_async(args=(i.rid,), queue=ACL_QUEUE)
rebuild.add(i.rid)
for _rid in rebuild:
role_rebuild.apply_async(args=(_rid, rg.app_id), queue=ACL_QUEUE)
ResourceGroupCache.clean(rg)
AuditCRUD.add_resource_log(rg.app_id, AuditOperateType.delete,
AuditScope.resource_group, rg.id, origin, {},
{'resource_ids': {'current': [], 'origin': existed_ids}, }
)
class ResourceCRUD(object):
cls = Resource
@staticmethod
def search(q, app_id, resource_type_id=None, page=1, page_size=None):
query = db.session.query(Resource).filter(
def _parse_resource_type_id(type_id, app_id):
try:
type_id = int(type_id)
except ValueError:
_type = ResourceType.get_by(name=type_id, app_id=app_id, first=True, to_dict=False)
type_id = _type and _type.id
return type_id
@classmethod
def search(cls, q, u, app_id, resource_type_id=None, page=1, page_size=None):
query = Resource.query.filter(
Resource.deleted.is_(False)).filter(Resource.app_id == app_id)
if q:
query = query.filter(Resource.name.ilike("%{0}%".format(q)))
if u and UserCache.get(u):
query = query.filter(Resource.uid == UserCache.get(u).uid)
if resource_type_id:
resource_type_id = cls._parse_resource_type_id(resource_type_id, app_id)
query = query.filter(Resource.resource_type_id == resource_type_id)
numfound = query.count()
res = [i.to_dict() for i in query.offset((page - 1) * page_size).limit(page_size)]
for i in res:
i['user'] = UserCache.get(i['uid']).nickname if i['uid'] else ''
return numfound, query.offset((page - 1) * page_size).limit(page_size)
return numfound, res
@staticmethod
def add(name, type_id, app_id):
Resource.get_by(name=name, resource_type_id=type_id, app_id=app_id) and abort(
400, "Resource <{0}> is already existed".format(name))
@classmethod
def add(cls, name, type_id, app_id, uid=None):
type_id = cls._parse_resource_type_id(type_id, app_id)
return Resource.create(name=name, resource_type_id=type_id, app_id=app_id)
Resource.get_by(name=name, resource_type_id=type_id, app_id=app_id) and \
abort(400, ErrFormat.resource_exists.format(name))
r = Resource.create(name=name, resource_type_id=type_id, app_id=app_id, uid=uid)
from api.tasks.acl import apply_trigger
triggers = TriggerCRUD.match_triggers(app_id, r.name, r.resource_type_id, uid)
current_app.logger.info(triggers)
for trigger in triggers:
# auto trigger should be no uid
apply_trigger.apply_async(args=(trigger.id,),
kwargs=dict(resource_id=r.id, ), queue=ACL_QUEUE)
AuditCRUD.add_resource_log(app_id, AuditOperateType.create,
AuditScope.resource, r.id, {}, r.to_dict(), {})
return r
@staticmethod
def update(_id, name):
resource = Resource.get_by_id(_id) or abort(404, "Resource <{0}> is not found".format(_id))
# todo trigger rebuild
resource = Resource.get_by_id(_id) or abort(404, ErrFormat.resource_not_found.format("id={}".format(_id)))
origin = resource.to_dict()
other = Resource.get_by(name=name, resource_type_id=resource.resource_type_id, to_dict=False, first=True)
if other and other.id != _id:
return abort(400, "Resource <{0}> is duplicated".format(name))
return abort(400, ErrFormat.resource_exists.format(name))
return resource.update(name=name)
ResourceCache.clean(resource)
resource = resource.update(name=name)
update_resource_to_build_role.apply_async(args=(_id, resource.app_id), queue=ACL_QUEUE)
AuditCRUD.add_resource_log(resource.app_id, AuditOperateType.update,
AuditScope.resource, resource.id, origin, resource.to_dict(), {})
return resource
@staticmethod
def delete(_id):
resource = Resource.get_by_id(_id) or abort(404, "Resource <{0}> is not found".format(_id))
resource = Resource.get_by_id(_id) or abort(404, ErrFormat.resource_not_found.format("id={}".format(_id)))
origin = resource.to_dict()
resource.soft_delete()
ResourceCache.clean(resource)
rebuilds = []
for i in RolePermission.get_by(resource_id=_id, to_dict=False):
i.soft_delete()
role_rebuild.apply_async(args=(i.rid,), queue=ACL_QUEUE)
rebuilds.append((i.rid, i.app_id))
for rid, app_id in set(rebuilds):
role_rebuild.apply_async(args=(rid, app_id), queue=ACL_QUEUE)
AuditCRUD.add_resource_log(resource.app_id, AuditOperateType.delete,
AuditScope.resource, resource.id, origin, {}, {})
@classmethod
def delete_by_name(cls, name, type_id, app_id):
resource = Resource.get_by(name=name, resource_type_id=type_id, app_id=app_id) or abort(
400, ErrFormat.resource_exists.format(name))
return cls.delete(resource.id)
@classmethod
def update_by_name(cls, name, type_id, app_id, new_name):
resource = Resource.get_by(name=name, resource_type_id=type_id, app_id=app_id) or abort(
400, ErrFormat.resource_exists.format(name))
return cls.update(resource.id, new_name)

42
cmdb-api/api/lib/perm/acl/resp_format.py Normal file
View File

@ -0,0 +1,42 @@
# -*- coding:utf-8 -*-
from api.lib.resp_format import CommonErrFormat
class ErrFormat(CommonErrFormat):
auth_only_with_app_token_failed = "应用 Token验证失败"
session_invalid = "您不是应用管理员 或者 session失效(尝试一下退出重新登录)"
resource_type_not_found = "资源类型 {} 不存在!"
resource_type_exists = "资源类型 {} 已经存在!"
resource_type_cannot_delete = "因为该类型下有资源的存在, 不能删除!"
user_not_found = "用户 {} 不存在!"
user_exists = "用户 {} 已经存在!"
role_not_found = "角色 {} 不存在!"
role_exists = "角色 {} 已经存在!"
global_role_not_found = "全局角色 {} 不存在!"
global_role_exists = "全局角色 {} 已经存在!"
resource_no_permission = "您没有资源: {} 的 {} 权限"
admin_required = "需要管理员权限"
role_required = "需要角色: {}"
app_is_ready_existed = "应用 {} 已经存在"
app_not_found = "应用 {} 不存在!"
app_secret_invalid = "应用的Secret无效"
resource_not_found = "资源 {} 不存在!"
resource_exists = "资源 {} 已经存在!"
resource_group_not_found = "资源组 {} 不存在!"
resource_group_exists = "资源组 {} 已经存在!"
inheritance_dead_loop = "继承检测到了死循环"
role_relation_not_found = "角色关系 {} 不存在!"
trigger_not_found = "触发器 {} 不存在!"
trigger_exists = "触发器 {} 已经存在!"
trigger_disabled = "触发器 {} 已经被禁用!"
invalid_password = "密码不正确!"

401
cmdb-api/api/lib/perm/acl/role.py
View File

@ -1,25 +1,39 @@
# -*- coding:utf-8 -*-
import time
import six
from flask import abort
from flask import current_app
from api.extensions import db
from api.lib.perm.acl.app import AppCRUD
from api.lib.perm.acl.audit import AuditCRUD, AuditOperateType, AuditScope
from api.lib.perm.acl.cache import AppCache
from api.lib.perm.acl.cache import HasResourceRoleCache
from api.lib.perm.acl.cache import RoleCache
from api.lib.perm.acl.cache import RoleRelationCache
from api.lib.perm.acl.cache import UserCache
from api.lib.perm.acl.const import ACL_QUEUE
from api.models.acl import Resource
from api.lib.perm.acl.const import OperateType
from api.lib.perm.acl.resource import ResourceGroupCRUD
from api.lib.perm.acl.resp_format import ErrFormat
from api.models.acl import Resource, ResourceGroup
from api.models.acl import ResourceGroupItems
from api.models.acl import ResourceType
from api.models.acl import Role
from api.models.acl import RolePermission
from api.models.acl import RoleRelation
from api.tasks.acl import op_record
from api.tasks.acl import role_rebuild
class RoleRelationCRUD(object):
cls = RoleRelation
@staticmethod
def get_parents(rids=None, uids=None):
def get_parents(rids=None, uids=None, app_id=None, all_app=False):
rid2uid = dict()
if uids is not None:
uids = [uids] if isinstance(uids, six.integer_types) else uids
@ -29,8 +43,22 @@ class RoleRelationCRUD(object):
else:
rids = [rids] if isinstance(rids, six.integer_types) else rids
res = db.session.query(RoleRelation).filter(
RoleRelation.child_id.in_(rids)).filter(RoleRelation.deleted.is_(False))
if app_id is not None:
res = db.session.query(RoleRelation).filter(
RoleRelation.child_id.in_(rids)).filter(RoleRelation.app_id == app_id).filter(
RoleRelation.deleted.is_(False)).union(
db.session.query(RoleRelation).filter(
RoleRelation.child_id.in_(rids)).filter(RoleRelation.app_id.is_(None)).filter(
RoleRelation.deleted.is_(False)))
elif not all_app:
res = db.session.query(RoleRelation).filter(
RoleRelation.child_id.in_(rids)).filter(RoleRelation.app_id.is_(None)).filter(
RoleRelation.deleted.is_(False))
else:
res = db.session.query(RoleRelation).filter(
RoleRelation.child_id.in_(rids)).filter(RoleRelation.deleted.is_(False))
id2parents = {}
for i in res:
id2parents.setdefault(rid2uid.get(i.child_id, i.child_id), []).append(RoleCache.get(i.parent_id).to_dict())
@ -38,24 +66,28 @@ class RoleRelationCRUD(object):
return id2parents
@staticmethod
def get_parent_ids(rid):
res = RoleRelation.get_by(child_id=rid, to_dict=False)
return [i.parent_id for i in res]
def get_parent_ids(rid, app_id):
if app_id is not None:
return [i.parent_id for i in RoleRelation.get_by(child_id=rid, app_id=app_id, to_dict=False)] + \
[i.parent_id for i in RoleRelation.get_by(child_id=rid, app_id=None, to_dict=False)]
else:
return [i.parent_id for i in RoleRelation.get_by(child_id=rid, app_id=app_id, to_dict=False)]
@staticmethod
def get_child_ids(rid):
res = RoleRelation.get_by(parent_id=rid, to_dict=False)
return [i.child_id for i in res]
def get_child_ids(rid, app_id):
if app_id is not None:
return [i.child_id for i in RoleRelation.get_by(parent_id=rid, app_id=app_id, to_dict=False)] + \
[i.child_id for i in RoleRelation.get_by(parent_id=rid, app_id=None, to_dict=False)]
else:
return [i.child_id for i in RoleRelation.get_by(parent_id=rid, app_id=app_id, to_dict=False)]
@classmethod
def recursive_parent_ids(cls, rid):
def recursive_parent_ids(cls, rid, app_id):
all_parent_ids = set()
def _get_parent(_id):
all_parent_ids.add(_id)
parent_ids = RoleRelationCache.get_parent_ids(_id)
parent_ids = RoleRelationCache.get_parent_ids(_id, app_id)
for parent_id in parent_ids:
_get_parent(parent_id)
@ -64,12 +96,12 @@ class RoleRelationCRUD(object):
return all_parent_ids
@classmethod
def recursive_child_ids(cls, rid):
def recursive_child_ids(cls, rid, app_id):
all_child_ids = set()
def _get_children(_id):
all_child_ids.add(_id)
child_ids = RoleRelationCache.get_child_ids(_id)
child_ids = RoleRelationCache.get_child_ids(_id, app_id)
for child_id in child_ids:
_get_children(child_id)
@ -78,55 +110,124 @@ class RoleRelationCRUD(object):
return all_child_ids
@classmethod
def add(cls, parent_id, child_id):
RoleRelation.get_by(parent_id=parent_id, child_id=child_id) and abort(400, "It's already existed")
def get_users_by_rid(cls, rid, app_id, rid2obj=None, uid2obj=None):
rid2obj = rid2obj or dict()
uid2obj = uid2obj or dict()
if parent_id in cls.recursive_child_ids(child_id):
return abort(400, "Circulation inheritance!!!")
users = []
rids = cls.recursive_child_ids(rid, app_id)
for rid in rids:
if rid not in rid2obj:
rid2obj[rid] = RoleCache.get(rid)
RoleRelationCache.clean(parent_id)
RoleRelationCache.clean(child_id)
role = rid2obj[rid]
if role and role.uid:
if role.uid and role.uid not in uid2obj:
uid2obj[role.uid] = UserCache.get(role.uid)
return RoleRelation.create(parent_id=parent_id, child_id=child_id)
u = uid2obj.get(role.uid)
u = u and u.to_dict()
if u:
u.update(dict(role=role.to_dict()))
users.append(u)
# todo role read log
# user_id = AuditCRUD.get_current_operate_uid()
# audit_role_log.apply_async(args=(app_id, user_id, result.copy()),
# queue=ACL_QUEUE)
return users
@classmethod
def delete(cls, _id):
existed = RoleRelation.get_by_id(_id) or abort(400, "RoleRelation <{0}> does not exist".format(_id))
child_ids = cls.recursive_child_ids(existed.child_id)
def add(cls, role, parent_id, child_ids, app_id):
result = []
for child_id in child_ids:
role_rebuild.apply_async(args=(child_id,), queue=ACL_QUEUE)
existed = RoleRelation.get_by(parent_id=parent_id, child_id=child_id, app_id=app_id)
if existed:
continue
RoleRelationCache.clean(existed.parent_id)
RoleRelationCache.clean(existed.child_id)
RoleRelationCache.clean(parent_id, app_id)
RoleRelationCache.clean(child_id, app_id)
if parent_id in cls.recursive_child_ids(child_id, app_id):
return abort(400, ErrFormat.inheritance_dead_loop)
if app_id is None:
for app in AppCRUD.get_all():
if app.name != "acl":
RoleRelationCache.clean(child_id, app.id)
result.append(RoleRelation.create(parent_id=parent_id, child_id=child_id, app_id=app_id).to_dict())
AuditCRUD.add_role_log(app_id, AuditOperateType.role_relation_add,
AuditScope.role_relation, role.id, {}, {},
{'child_ids': list(child_ids), 'parent_ids': [parent_id], }
)
return result
@classmethod
def delete(cls, _id, app_id):
existed = RoleRelation.get_by_id(_id) or abort(
400, ErrFormat.role_relation_not_found.format("id={}".format(_id)))
child_ids = cls.recursive_child_ids(existed.child_id, app_id)
for child_id in child_ids:
role_rebuild.apply_async(args=(child_id, app_id), queue=ACL_QUEUE)
role = RoleCache.get(existed.parent_id)
existed.soft_delete()
RoleRelationCache.clean(existed.parent_id, app_id)
RoleRelationCache.clean(existed.child_id, app_id)
AuditCRUD.add_role_log(app_id, AuditOperateType.role_relation_delete,
AuditScope.role_relation, role.id, {}, {},
{'child_ids': list(child_ids), 'parent_ids': [existed.parent_id], }
)
@classmethod
def delete2(cls, parent_id, child_id):
existed = RoleRelation.get_by(parent_id=parent_id, child_id=child_id, first=True, to_dict=False)
existed or abort(400, "RoleRelation < {0} -> {1} > does not exist".format(parent_id, child_id))
def delete2(cls, parent_id, child_id, app_id):
existed = RoleRelation.get_by(parent_id=parent_id, child_id=child_id, app_id=app_id, first=True, to_dict=False)
existed or abort(400, ErrFormat.role_relation_not_found.format("{} -> {}".format(parent_id, child_id)))
child_ids = cls.recursive_child_ids(existed.child_id)
for child_id in child_ids:
role_rebuild.apply_async(args=(child_id,), queue=ACL_QUEUE)
RoleRelationCache.clean(existed.parent_id)
RoleRelationCache.clean(existed.child_id)
role = RoleCache.get(existed.parent_id)
existed.soft_delete()
child_ids = cls.recursive_child_ids(existed.child_id, app_id)
for child_id in child_ids:
role_rebuild.apply_async(args=(child_id, app_id), queue=ACL_QUEUE)
RoleRelationCache.clean(existed.parent_id, app_id)
RoleRelationCache.clean(existed.child_id, app_id)
AuditCRUD.add_role_log(app_id, AuditOperateType.role_relation_delete,
AuditScope.role_relation, role.id, {}, {},
{'child_ids': list(child_ids), 'parent_ids': [existed.parent_id], }
)
class RoleCRUD(object):
cls = Role
@staticmethod
def search(q, app_id, page=1, page_size=None, user_role=True):
def search(q, app_id, page=1, page_size=None, user_role=True, is_all=False, user_only=False):
query = db.session.query(Role).filter(Role.deleted.is_(False))
query = query.filter(Role.app_id == app_id).filter(Role.uid.is_(None))
query1 = query.filter(Role.app_id == app_id).filter(Role.uid.is_(None))
query2 = query.filter(Role.app_id.is_(None)).filter(Role.uid.is_(None))
query = query1.union(query2)
if user_role:
query1 = db.session.query(Role).filter(Role.deleted.is_(False)).filter(Role.uid.isnot(None))
query = query.union(query1)
if user_only:
query = db.session.query(Role).filter(Role.deleted.is_(False)).filter(Role.uid.isnot(None))
if not is_all:
role_ids = list(HasResourceRoleCache.get(app_id).keys())
query = query.filter(Role.id.in_(role_ids))
if q:
query = query.filter(Role.name.ilike('%{0}%'.format(q)))
@ -135,46 +236,99 @@ class RoleCRUD(object):
return numfound, query.offset((page - 1) * page_size).limit(page_size)
@staticmethod
def add_role(name, app_id=None, is_app_admin=False, uid=None):
Role.get_by(name=name, app_id=app_id) and abort(400, "Role <{0}> is already existed".format(name))
def add_role(name, app_id=None, is_app_admin=False, uid=None, password=None):
if app_id and AppCache.get(app_id).name == "acl":
app_id = None
return Role.create(name=name,
Role.get_by(name=name, app_id=app_id) and abort(400, ErrFormat.role_exists.format(name))
if app_id is not None:
Role.get_by(name=name, app_id=None) and abort(400, ErrFormat.global_role_exists.format(name))
from api.lib.perm.acl.user import UserCRUD
key, secret = UserCRUD.gen_key_secret()
role = Role.create(name=name,
app_id=app_id,
is_app_admin=is_app_admin,
password=password,
key=key,
secret=secret,
uid=uid)
AuditCRUD.add_role_log(app_id, AuditOperateType.create,
AuditScope.role, role.id, {}, role.to_dict(), {})
return role
@staticmethod
def update_role(rid, **kwargs):
kwargs.pop('app_id', None)
role = Role.get_by_id(rid) or abort(404, "Role <{0}> does not exist".format(rid))
role = Role.get_by_id(rid) or abort(404, ErrFormat.role_not_found.format("rid={}".format(rid)))
origin = role.to_dict()
RoleCache.clean(rid)
return role.update(**kwargs)
role = role.update(**kwargs)
AuditCRUD.add_role_log(role.app_id, AuditOperateType.update,
AuditScope.role, role.id, origin, role.to_dict(), {},
)
return role
@staticmethod
def get_by_name(name, app_id):
role = Role.get_by(name=name, app_id=app_id)
return role
@classmethod
def delete_role(cls, rid):
role = Role.get_by_id(rid) or abort(404, "Role <{0}> does not exist".format(rid))
from api.lib.perm.acl.acl import is_admin
role = Role.get_by_id(rid) or abort(404, ErrFormat.role_not_found.format("rid={}".format(rid)))
if not role.app_id and not is_admin():
return abort(403, ErrFormat.admin_required)
origin = role.to_dict()
child_ids = []
parent_ids = []
recursive_child_ids = list(RoleRelationCRUD.recursive_child_ids(rid, role.app_id))
for i in RoleRelation.get_by(parent_id=rid, to_dict=False):
child_ids.append(i.child_id)
i.soft_delete()
for i in RoleRelation.get_by(child_id=rid, to_dict=False):
parent_ids.append(i.parent_id)
i.soft_delete()
role_permissions = []
for i in RolePermission.get_by(rid=rid, to_dict=False):
role_permissions.append(i.to_dict())
i.soft_delete()
role_rebuild.apply_async(args=(list(RoleRelationCRUD.recursive_child_ids(rid)), ), queue=ACL_QUEUE)
RoleCache.clean(rid)
RoleRelationCache.clean(rid)
role.soft_delete()
role_rebuild.apply_async(args=(recursive_child_ids, role.app_id), queue=ACL_QUEUE)
RoleCache.clean(rid)
RoleRelationCache.clean(rid, role.app_id)
AuditCRUD.add_role_log(role.app_id, AuditOperateType.delete,
AuditScope.role, role.id, origin, {},
{'child_ids': child_ids, 'parent_ids': parent_ids,
'role_permissions': role_permissions, },
)
@staticmethod
def get_resources(rid):
res = RolePermission.get_by(rid=rid, to_dict=False)
def get_resources(rid, app_id):
res = RolePermission.get_by(rid=rid, app_id=app_id, to_dict=False)
id2perms = dict(id2perms={}, group2perms={})
for i in res:
if i.resource_id:
@ -184,24 +338,90 @@ class RoleCRUD(object):
return id2perms
@staticmethod
def _extend_resources(rid, resource_type_id, app_id):
res = RoleRelationCache.get_resources2(rid, app_id)
resources = {_id: res['resources'][_id] for _id in res['resources']
if not resource_type_id or resource_type_id == res['resources'][_id]['resource_type_id']}
groups = {_id: res['groups'][_id] for _id in res['groups']
if not resource_type_id or resource_type_id == res['groups'][_id]['resource_type_id']}
return resources, groups
@classmethod
def recursive_resources(cls, rid, app_id, resource_type_id=None, group_flat=True, to_record=False):
def _merge(a, b):
for i in b:
if i in a:
a[i]['permissions'] = list(set(a[i]['permissions'] + b[i]['permissions']))
else:
a[i] = b[i]
return a
try:
resource_type_id = resource_type_id and int(resource_type_id)
except ValueError:
resource_type = ResourceType.get_by(name=resource_type_id, app_id=app_id, first=True, to_dict=False)
resource_type_id = resource_type and resource_type.id
result = dict(resources=dict(), groups=dict())
s = time.time()
parent_ids = RoleRelationCRUD.recursive_parent_ids(rid, app_id)
current_app.logger.info('parent ids {0}: {1}'.format(parent_ids, time.time() - s))
for parent_id in parent_ids:
_resources, _groups = cls._extend_resources(parent_id, resource_type_id, app_id)
current_app.logger.info('middle1: {0}'.format(time.time() - s))
_merge(result['resources'], _resources)
current_app.logger.info('middle2: {0}'.format(time.time() - s))
current_app.logger.info(len(_groups))
if not group_flat:
_merge(result['groups'], _groups)
else:
for rg_id in _groups:
items = ResourceGroupCRUD.get_items(rg_id)
for item in items:
if not resource_type_id or resource_type_id == item['resource_type_id']:
item.setdefault('permissions', [])
item['permissions'] = list(set(item['permissions'] + _groups[rg_id]['permissions']))
result['resources'][item['id']] = item
current_app.logger.info('End: {0}'.format(time.time() - s))
result['resources'] = list(result['resources'].values())
result['groups'] = list(result['groups'].values())
if to_record:
op_record.apply_async(args=(app_id, rid, OperateType.READ, ["resources"]),
queue=ACL_QUEUE)
# todo role read log
# user_id = AuditCRUD.get_current_operate_uid()
# audit_role_log.apply_async(args=(app_id, user_id, result.copy()),
# queue=ACL_QUEUE)
return result
@staticmethod
def get_group_ids(resource_id):
return [i.group_id for i in ResourceGroupItems.get_by(resource_id=resource_id, to_dict=False)]
@classmethod
def has_permission(cls, rid, resource_name, resource_type, app_id, perm):
resource_type = ResourceType.get_by(app_id=app_id, name=resource_type, first=True, to_dict=False)
resource_type or abort(404, "ResourceType <{0}> is not found".format(resource_type))
type_id = resource_type.id
resource = Resource.get_by(name=resource_name, resource_type_id=type_id, first=True, to_dict=False)
resource = resource or abort(403, "Resource <{0}> is not in ACL".format(resource_name))
def has_permission(cls, rid, resource_name, resource_type_name, app_id, perm, resource_id=None):
current_app.logger.debug((rid, resource_name, resource_type_name, app_id, perm))
if not resource_id:
resource_type = ResourceType.get_by(app_id=app_id, name=resource_type_name, first=True, to_dict=False)
resource_type or abort(404, ErrFormat.resource_type_not_found.format(resource_type_name))
type_id = resource_type.id
resource = Resource.get_by(name=resource_name, resource_type_id=type_id, first=True, to_dict=False)
resource = resource or abort(403, ErrFormat.resource_not_found.format(resource_name))
resource_id = resource.id
parent_ids = RoleRelationCRUD.recursive_parent_ids(rid)
group_ids = cls.get_group_ids(resource.id)
parent_ids = RoleRelationCRUD.recursive_parent_ids(rid, app_id)
group_ids = cls.get_group_ids(resource_id)
for parent_id in parent_ids:
id2perms = RoleRelationCache.get_resources(parent_id)
perms = id2perms['id2perms'].get(resource.id, [])
id2perms = RoleRelationCache.get_resources(parent_id, app_id)
current_app.logger.debug(id2perms)
perms = id2perms['id2perms'].get(resource_id, [])
if perms and {perm}.issubset(set(perms)):
return True
@ -213,19 +433,60 @@ class RoleCRUD(object):
return False
@classmethod
def get_permissions(cls, rid, resource_name):
resource = Resource.get_by(name=resource_name, first=True, to_dict=False)
resource = resource or abort(403, "Resource <{0}> is not in ACL".format(resource_name))
def get_permissions(cls, rid, resource_name, app_id):
parent_ids = RoleRelationCRUD.recursive_parent_ids(rid)
resource = Resource.get_by(name=resource_name, first=True, to_dict=False)
resource = resource or abort(403, ErrFormat.resource_not_found.format(resource_name))
parent_ids = RoleRelationCRUD.recursive_parent_ids(rid, app_id)
group_ids = cls.get_group_ids(resource.id)
perms = []
for parent_id in parent_ids:
id2perms = RoleRelationCache.get_resources(parent_id)
id2perms = RoleRelationCache.get_resources(parent_id, app_id)
perms += id2perms['id2perms'].get(parent_id, [])
for group_id in group_ids:
perms += id2perms['group2perms'].get(group_id, [])
return set(perms)
@classmethod
def list_resources(cls, app_id, rids, resource_type_id=None, q=None):
query = db.session.query(Resource, RolePermission).filter(
Resource.app_id == app_id,
Resource.deleted.is_(False),
RolePermission.deleted.is_(False),
RolePermission.rid.in_(rids),
).join(
RolePermission, Resource.id == RolePermission.resource_id
)
if resource_type_id:
query = query.filter(Resource.resource_type_id == resource_type_id)
if q:
query = query.filter(Resource.resource_type_id == resource_type_id)
return query.all()
@classmethod
def list_resource_groups(cls, app_id, rids, resource_type_id=None, q=None):
query = db.session.query(ResourceGroup, RolePermission).filter(
ResourceGroup.app_id == app_id,
ResourceGroup.deleted.is_(False),
RolePermission.deleted.is_(False),
RolePermission.rid.in_(rids),
).join(
RolePermission, ResourceGroup.id == RolePermission.group_id
)
if resource_type_id:
query = query.filter(ResourceGroup.resource_type_id == resource_type_id)
if q:
query = query.filter(ResourceGroup.resource_type_id == resource_type_id)
return query.all()

162
cmdb-api/api/lib/perm/acl/trigger.py Normal file
View File

@ -0,0 +1,162 @@
# -*- coding:utf-8 -*-
import copy
import json
import re
from fnmatch import fnmatch
from flask import abort, current_app
from api.lib.perm.acl.audit import AuditCRUD, AuditOperateType
from api.lib.perm.acl.cache import UserCache
from api.lib.perm.acl.const import ACL_QUEUE
from api.lib.perm.acl.resp_format import ErrFormat
from api.models.acl import Trigger
from api.tasks.acl import apply_trigger, cancel_trigger
class TriggerCRUD(object):
cls = Trigger
@staticmethod
def get(app_id):
triggers = Trigger.get_by(app_id=app_id)
for trigger in triggers:
trigger['uid'] = json.loads(trigger['uid'] or '[]')
trigger['users'] = [UserCache.get(i).username for i in trigger['uid']]
trigger['roles'] = json.loads(trigger['roles'] or '[]')
trigger['permissions'] = json.loads(trigger['permissions'] or '[]')
return triggers
@staticmethod
def add(app_id, **kwargs):
kwargs.pop('app_id', None)
kwargs['roles'] = json.dumps(kwargs['roles'] or [])
kwargs['permissions'] = json.dumps(kwargs['permissions'] or [])
kwargs['uid'] = json.dumps(kwargs.get('uid') or [])
_kwargs = copy.deepcopy(kwargs)
_kwargs.pop('name', None)
Trigger.get_by(app_id=app_id, **_kwargs) and abort(400, ErrFormat.trigger_exists.format(""))
t = Trigger.create(app_id=app_id, **kwargs)
AuditCRUD.add_trigger_log(app_id, t.id, AuditOperateType.create, {}, t.to_dict(), {})
return t
@staticmethod
def update(_id, **kwargs):
existed = Trigger.get_by_id(_id) or abort(404, ErrFormat.trigger_not_found.format("id={}".format(_id)))
origin = existed.to_dict()
kwargs['roles'] = json.dumps(kwargs['roles'] or [])
kwargs['uid'] = json.dumps(kwargs['uid'] or [])
kwargs['permissions'] = json.dumps(kwargs['permissions'] or [])
existed.update(**kwargs)
AuditCRUD.add_trigger_log(existed.app_id, existed.id, AuditOperateType.update,
origin, existed.to_dict(), {})
return existed
@staticmethod
def delete(_id):
existed = Trigger.get_by_id(_id) or abort(404, ErrFormat.trigger_not_found.format("id={}".format(_id)))
origin = existed.to_dict()
existed.soft_delete()
AuditCRUD.add_trigger_log(existed.app_id, existed.id, AuditOperateType.delete,
origin, {}, {}
)
return existed
@staticmethod
def apply(_id):
trigger = Trigger.get_by_id(_id) or abort(404, ErrFormat.trigger_not_found.format("id={}".format(_id)))
if not trigger.enabled:
return abort(400, ErrFormat.trigger_disabled.format("id={}".format(_id)))
user_id = AuditCRUD.get_current_operate_uid()
apply_trigger.apply_async(args=(_id,), kwargs=dict(operator_uid=user_id), queue=ACL_QUEUE)
@staticmethod
def cancel(_id):
trigger = Trigger.get_by_id(_id) or abort(404, ErrFormat.trigger_not_found.format("id={}".format(_id)))
if not trigger.enabled:
return abort(400, ErrFormat.trigger_disabled.format("id={}".format(_id)))
user_id = AuditCRUD.get_current_operate_uid()
cancel_trigger.apply_async(args=(_id,), kwargs=dict(operator_uid=user_id), queue=ACL_QUEUE)
@staticmethod
def match_triggers(app_id, resource_name, resource_type_id, uid):
triggers = Trigger.get_by(app_id=app_id, enabled=True, resource_type_id=resource_type_id, to_dict=False)
def _fnmatch(name, wildcard):
import re
try:
return re.compile(wildcard).findall(name)
except:
return fnmatch(name, trigger.wildcard)
uid = int(uid) if uid else uid
_match_triggers = []
for trigger in triggers:
uids = json.loads(trigger.uid or '[]')
if trigger.wildcard and uids:
if _fnmatch(resource_name, trigger.wildcard) and uid in uids:
_match_triggers.append(trigger)
elif trigger.wildcard:
if _fnmatch(resource_name, trigger.wildcard):
_match_triggers.append(trigger)
elif uids:
if uid in uids:
_match_triggers.append(trigger)
return _match_triggers
@staticmethod
def get_resources(app_id, resource_type_id, wildcard, uid):
from api.models.acl import Resource
wildcard = wildcard or ''
if wildcard and uid:
query = Resource.get_by(__func_in___key_uid=uid,
app_id=app_id,
resource_type_id=resource_type_id,
only_query=True)
try:
re.compile(wildcard)
resources = query.filter(Resource.name.op('regexp')(wildcard)).all()
except:
resources = query.filter(Resource.name.ilike(wildcard.replace('*', '%'))).all()
elif wildcard:
query = Resource.get_by(app_id=app_id,
resource_type_id=resource_type_id,
only_query=True)
try:
re.compile(wildcard)
resources = query.filter(Resource.name.op('regexp')(wildcard)).all()
except:
resources = query.filter(Resource.name.ilike(wildcard.replace('*', '%'))).all()
elif uid:
resources = Resource.get_by(__func_in___key_uid=uid,
app_id=app_id,
resource_type_id=resource_type_id,
to_dict=False)
else:
resources = []
return resources

68
cmdb-api/api/lib/perm/acl/user.py
View File

@ -9,13 +9,17 @@ from flask import abort
from flask import g
from api.extensions import db
from api.lib.perm.acl.audit import AuditCRUD, AuditOperateType, AuditScope
from api.lib.perm.acl.cache import UserCache
from api.lib.perm.acl.resp_format import ErrFormat
from api.lib.perm.acl.role import RoleCRUD
from api.models.acl import Role
from api.models.acl import User
class UserCRUD(object):
cls = User
@staticmethod
def search(q, page=1, page_size=None):
query = db.session.query(User).filter(User.deleted.is_(False))
@ -27,7 +31,7 @@ class UserCRUD(object):
return numfound, query.offset((page - 1) * page_size).limit(page_size)
@staticmethod
def _gen_key_secret():
def gen_key_secret():
key = uuid.uuid4().hex
secret = ''.join(random.sample(string.ascii_letters + string.digits + '~!@#$%^&*?', 32))
@ -36,62 +40,76 @@ class UserCRUD(object):
@classmethod
def add(cls, **kwargs):
existed = User.get_by(username=kwargs['username'], email=kwargs['email'])
existed and abort(400, "User <{0}> is already existed".format(kwargs['username']))
existed and abort(400, ErrFormat.user_exists.format(kwargs['username']))
is_admin = kwargs.pop('is_admin', False)
kwargs['nickname'] = kwargs.get('nickname') or kwargs['username']
kwargs['block'] = 0
kwargs['key'], kwargs['secret'] = cls._gen_key_secret()
kwargs['key'], kwargs['secret'] = cls.gen_key_secret()
user_employee = db.session.query(User).filter(User.deleted.is_(False)).order_by(
User.employee_id.desc()).first()
biggest_employee_id = int(float(user_employee.employee_id)) \
if user_employee is not None else 0
kwargs['employee_id'] = '{0:04d}'.format(biggest_employee_id + 1)
user = User.create(**kwargs)
role = RoleCRUD.add_role(user.username, uid=user.uid)
if is_admin:
from api.lib.perm.acl.cache import AppCache
from api.lib.perm.acl.role import RoleRelationCRUD
admin_r = Role.get_by(name='admin', first=True, to_dict=False) or \
RoleCRUD.add_role('admin', AppCache.get('cmdb').id, True)
RoleRelationCRUD.add(admin_r.id, role.id)
RoleCRUD.add_role(user.username, uid=user.uid)
AuditCRUD.add_role_log(None, AuditOperateType.create,
AuditScope.user, user.uid, {}, user.to_dict(), {}, {}
)
return user
@staticmethod
def update(uid, **kwargs):
user = User.get_by(uid=uid, to_dict=False, first=True) or abort(404, "User <{0}> does not exist".format(uid))
user = User.get_by(uid=uid, to_dict=False, first=True) or abort(
404, ErrFormat.user_not_found.format("uid={}".format(uid)))
if kwargs.get("username"):
other = User.get_by(username=kwargs['username'], first=True, to_dict=False)
if other is not None and other.uid != user.uid:
return abort(400, "User <{0}> cannot be duplicated".format(kwargs['username']))
return abort(400, ErrFormat.user_exists.format(kwargs['username']))
UserCache.clean(user)
origin = user.to_dict()
if kwargs.get("username") and kwargs['username'] != user.username:
role = Role.get_by(name=user.username, first=True, to_dict=False)
if role is not None:
RoleCRUD.update_role(role.id, **dict(name=kwargs['name']))
RoleCRUD.update_role(role.id, **dict(name=kwargs['username']))
return user.update(**kwargs)
user = user.update(**kwargs)
AuditCRUD.add_role_log(None, AuditOperateType.update,
AuditScope.user, user.uid, origin, user.to_dict(), {}, {}
)
return user
@classmethod
def reset_key_secret(cls):
key, secret = cls._gen_key_secret()
key, secret = cls.gen_key_secret()
g.user.update(key=key, secret=secret)
UserCache.clean(g.user)
return key, secret
@classmethod
def delete(cls, uid):
if hasattr(g, 'user') and uid == g.user.uid:
return abort(400, "You cannot delete yourself")
user = User.get_by(uid=uid, to_dict=False, first=True) or abort(
404, ErrFormat.user_not_found.format("uid={}".format(uid)))
user = User.get_by(uid=uid, to_dict=False, first=True) or abort(404, "User <{0}> does not exist".format(uid))
origin = user.to_dict()
user.soft_delete()
UserCache.clean(user)
for i in Role.get_by(uid=uid, to_dict=False):
i.delete()
AuditCRUD.add_role_log(None, AuditOperateType.delete,
AuditScope.user, user.uid, origin, {}, {}, {})
user.delete()
@staticmethod
def get_employees():
return User.get_by(__func_isnot__key_employee_id=None, to_dict=True)

151
cmdb-api/api/lib/perm/auth.py
View File

@ -1,5 +1,4 @@
# -*- coding:utf-8 -*-
# -*- coding:utf-8 -*-
from __future__ import unicode_literals
@ -13,20 +12,54 @@ from flask import request
from flask import session
from flask_login import login_user
from api.lib.perm.acl.acl import ACLManager
from api.lib.perm.acl.acl import is_app_admin
from api.lib.perm.acl.cache import AppCache
from api.lib.perm.acl.cache import UserCache
from api.lib.perm.acl.resp_format import ErrFormat
from api.models.acl import Role
from api.models.acl import User
def reset_session(user, role=None):
from api.lib.perm.acl.acl import ACLManager
if role is not None:
user_info = ACLManager.get_user_info(role)
else:
user_info = ACLManager.get_user_info(user.username)
session["acl"] = dict(uid=user_info.get("uid"),
avatar=user.avatar if user else user_info.get("avatar"),
userId=user_info.get("uid"),
userName=user_info.get("username"),
nickName=user_info.get("nickname"),
parentRoles=user_info.get("parents"),
childRoles=user_info.get("children"),
roleName=user_info.get("role"))
session["uid"] = user_info.get("uuid")
def _auth_with_key():
key = request.values.get('_key')
secret = request.values.get('_secret')
if not key:
return False
path = request.path
keys = sorted(request.values.keys())
req_args = [request.values[k] for k in keys if k not in ("_key", "_secret")]
req_args = [str(request.values[k]) for k in keys if k not in ("_key", "_secret") and
not isinstance(request.values[k], (dict, list))]
user, authenticated = User.query.authenticate_with_key(key, secret, req_args, path)
if user and authenticated:
login_user(user)
reset_session(user)
return True
role, authenticated = Role.query.authenticate_with_key(key, secret, req_args, path)
if role and authenticated:
reset_session(None, role=role.name)
return True
return False
@ -53,19 +86,20 @@ def _auth_with_token():
return False
login_user(user)
g.user = user
reset_session(user)
return True
except jwt.ExpiredSignatureError:
return False
except (jwt.InvalidTokenError, Exception):
except (jwt.InvalidTokenError, Exception) as e:
current_app.logger.error(str(e))
return False
def _auth_with_ip_white_list():
ip = request.remote_addr
ip = request.headers.get('X-Real-IP') or request.remote_addr
key = request.values.get('_key')
secret = request.values.get('_secret')
current_app.logger.info(ip)
if not key and not secret and ip.strip() in current_app.config.get("WHITE_LIST", []): # TODO
user = UserCache.get("worker")
login_user(user)
@ -73,24 +107,87 @@ def _auth_with_ip_white_list():
return False
def _auth_with_app_token():
if _auth_with_session():
if not is_app_admin(request.values.get('app_id')) and request.method != "GET":
return False
elif is_app_admin(request.values.get('app_id')):
return True
if _auth_with_key() and is_app_admin('acl'):
return True
auth_headers = request.headers.get('App-Access-Token', '').strip()
if not auth_headers:
return False
try:
token = auth_headers
data = jwt.decode(token, current_app.config['SECRET_KEY'], algorithms=['HS256'])
current_app.logger.warning(data)
app = AppCache.get(data['sub'])
if not app:
return False
request.values['app_id'] = app.id
return True
except jwt.ExpiredSignatureError:
return False
except (jwt.InvalidTokenError, Exception) as e:
current_app.logger.error(str(e))
return False
def _auth_with_acl_token():
token = request.headers.get('Authorization', "")
if not token.startswith('Bearer '):
abort(401, ErrFormat.unauthorized)
_token = token.split(' ')[-1]
result = ACLManager().authenticate_with_token(_token)
if result.get('authenticated') and result.get('user'):
user = User.query.filter_by(email=result.get("user", {}).get("email", "")).first()
login_user(user)
reset_session(user)
return user
elif result.get('authenticated') is False:
abort(401, ErrFormat.unauthorized)
def auth_required(func):
if request.json is not None:
setattr(request, 'values', request.json)
else:
setattr(request, 'values', request.values.to_dict())
current_app.logger.debug(request.values)
@wraps(func)
def wrapper(*args, **kwargs):
if not getattr(func, 'authenticated', True):
return func(*args, **kwargs)
if _auth_with_session() or _auth_with_key() or _auth_with_token() or _auth_with_ip_white_list():
if getattr(func, 'auth_only_with_app_token', False) and _auth_with_app_token():
return func(*args, **kwargs)
elif getattr(func, 'auth_only_with_app_token', False):
if _auth_with_key() and is_app_admin('acl'):
return func(*args, **kwargs)
if request.headers.get('App-Access-Token', '').strip():
return abort(403, ErrFormat.auth_only_with_app_token_failed)
else:
return abort(403, ErrFormat.session_invalid)
if getattr(func, 'auth_with_app_token', False) and _auth_with_app_token():
return func(*args, **kwargs)
abort(401)
elif _auth_with_session() or _auth_with_key() or _auth_with_token() or _auth_with_ip_white_list():
return func(*args, **kwargs)
if _auth_with_acl_token():
return func(*args, **kwargs)
return abort(401, ErrFormat.unauthorized)
return wrapper
@ -103,3 +200,33 @@ def auth_abandoned(func):
return func(*args, **kwargs)
return wrapper
def auth_with_app_token(func):
setattr(func, 'auth_with_app_token', True)
@wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
def auth_only_for_acl(func):
setattr(func, 'auth_only_with_app_token', True)
@wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
def auth_with_acl_token(func):
setattr(func, 'auth_with_acl_token', True)
@wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper()

27
cmdb-api/api/lib/resp_format.py Normal file
View File

@ -0,0 +1,27 @@
# -*- coding:utf-8 -*-
class CommonErrFormat(object):
unauthorized = "未认证"
unknown_error = "未知错误"
invalid_request = "不合法的请求"
invalid_operation = "无效的操作"
not_found = "不存在"
unknown_search_error = "未知搜索错误"
invalid_json = "json格式似乎不正确了, 请仔细确认一下!"
datetime_argument_invalid = "参数 {} 格式不正确, 格式必须是: yyyy-mm-dd HH:MM:SS"
argument_value_required = "参数 {} 的值不能为空!"
argument_required = "请求缺少参数 {}"
argument_invalid = "参数 {} 的值无效"
argument_str_length_limit = "参数 {} 的长度必须 <= {}"
role_required = "角色 {} 才能操作!"
user_not_found = "用户 {} 不存在"
no_permission = "您没有资源: {} 的{}权限!"
no_permission2 = "您没有操作权限!"
no_permission_only_owner = "只有创建人或者管理员才有权限!"

262
cmdb-api/api/lib/utils.py
View File

@ -1,39 +1,19 @@
# -*- coding:utf-8 -*-
import base64
import json
import sys
import time
from typing import Set
import elasticsearch
import redis
import six
from Crypto.Cipher import AES
from elasticsearch import Elasticsearch
from flask import current_app
def get_page(page):
try:
page = int(page)
except ValueError:
page = 1
return page if page >= 1 else 1
def get_page_size(page_size):
if page_size == "all":
return page_size
try:
page_size = int(page_size)
except (ValueError, TypeError):
page_size = current_app.config.get("DEFAULT_PAGE_COUNT")
return page_size if page_size >= 1 else current_app.config.get("DEFAULT_PAGE_COUNT")
def handle_arg_list(arg):
if isinstance(arg, six.string_types) and arg.startswith('['):
return json.loads(arg)
return list(filter(lambda x: x != "", arg.strip().split(","))) if isinstance(arg, six.string_types) else arg
class BaseEnum(object):
_ALL_ = set() # type: Set[str]
@ -52,6 +32,46 @@ class BaseEnum(object):
return cls._ALL_
def get_page(page):
try:
page = int(page)
except (TypeError, ValueError):
page = 1
return page if page >= 1 else 1
def get_page_size(page_size):
if page_size == "all":
return page_size
try:
page_size = int(page_size)
except (ValueError, TypeError):
page_size = current_app.config.get("DEFAULT_PAGE_COUNT")
return page_size if page_size >= 1 else current_app.config.get("DEFAULT_PAGE_COUNT")
def handle_bool_arg(arg):
if arg in current_app.config.get("BOOL_TRUE"):
return True
return False
def handle_arg_list(arg):
if isinstance(arg, (list, dict)):
return arg
if arg == 0:
return [0]
if not arg:
return []
if isinstance(arg, (six.integer_types, float)):
return [arg]
return list(filter(lambda x: x != "", arg.strip().split(","))) if isinstance(arg, six.string_types) else arg
class RedisHandler(object):
def __init__(self, flask_app=None):
self.flask_app = flask_app
@ -65,7 +85,8 @@ class RedisHandler(object):
max_connections=config.get("REDIS_MAX_CONN"),
host=config.get("CACHE_REDIS_HOST"),
port=config.get("CACHE_REDIS_PORT"),
db=config.get("REDIS_DB"))
password=config.get("CACHE_REDIS_PASSWORD"),
db=config.get("REDIS_DB") or 0)
self.r = redis.Redis(connection_pool=pool)
except Exception as e:
current_app.logger.warning(str(e))
@ -106,9 +127,23 @@ class ESHandler(object):
def init_app(self, app):
self.flask_app = app
config = self.flask_app.config
self.es = Elasticsearch(config.get("ES_HOST"))
if not self.es.indices.exists(index=self.index):
self.es.indices.create(index=self.index)
if config.get('ES_USER') and config.get('ES_PASSWORD'):
uri = "http://{}:{}@{}:{}/".format(config.get('ES_USER'), config.get('ES_PASSWORD'),
config.get('ES_HOST'), config.get('ES_PORT'))
else:
uri = "{}:{}".format(config.get('ES_HOST'), config.get('ES_PORT') or 9200)
self.es = Elasticsearch(uri,
timeout=10,
max_retries=3,
retry_on_timeout=True,
retry_on_status=(502, 503, 504, "N/A"),
maxsize=10)
try:
if not self.es.indices.exists(index=self.index):
self.es.indices.create(index=self.index)
except elasticsearch.exceptions.RequestError as ex:
if ex.error != 'resource_already_exists_exception':
raise
def update_mapping(self, field, value_type, other):
body = {
@ -123,6 +158,12 @@ class ESHandler(object):
)
def get_index_id(self, ci_id):
try:
return self._get_index_id(ci_id)
except:
return self._get_index_id(ci_id)
def _get_index_id(self, ci_id):
query = {
'query': {
'match': {'ci_id': ci_id}
@ -168,3 +209,164 @@ class ESHandler(object):
res.get("aggregations", {})
else:
return 0, [], {}
class Lock(object):
def __init__(self, name, timeout=10, app=None, need_lock=True):
self.lock_key = name
self.need_lock = need_lock
self.timeout = timeout
if not app:
app = current_app
self.app = app
try:
self.redis = redis.Redis(host=self.app.config.get('CACHE_REDIS_HOST'),
port=self.app.config.get('CACHE_REDIS_PORT'),
password=self.app.config.get('CACHE_REDIS_PASSWORD'))
except:
self.app.logger.error("cannot connect redis")
raise Exception("cannot connect redis")
def lock(self, timeout=None):
if not timeout:
timeout = self.timeout
retry = 0
while retry < 100:
timestamp = time.time() + timeout + 1
_lock = self.redis.setnx(self.lock_key, timestamp)
if _lock == 1 or (
time.time() > float(self.redis.get(self.lock_key) or sys.maxsize) and
time.time() > float(self.redis.getset(self.lock_key, timestamp) or sys.maxsize)):
break
else:
retry += 1
time.sleep(0.6)
if retry >= 100:
raise Exception("get lock failed...")
def release(self):
if time.time() < float(self.redis.get(self.lock_key)):
self.redis.delete(self.lock_key)
def __enter__(self):
if self.need_lock:
self.lock()
def __exit__(self, exc_type, exc_val, exc_tb):
if self.need_lock:
self.release()
class Redis2Handler(object):
def __init__(self, flask_app=None, prefix=None):
self.flask_app = flask_app
self.prefix = prefix
self.r = None
def init_app(self, app):
self.flask_app = app
config = self.flask_app.config
try:
pool = redis.ConnectionPool(
max_connections=config.get("REDIS_MAX_CONN"),
host=config.get("ONEAGENT_REDIS_HOST"),
port=config.get("ONEAGENT_REDIS_PORT"),
db=config.get("ONEAGENT_REDIS_DB"),
password=config.get("ONEAGENT_REDIS_PASSWORD")
)
self.r = redis.Redis(connection_pool=pool)
except Exception as e:
current_app.logger.warning(str(e))
current_app.logger.error("init redis connection failed")
def get(self, key):
try:
value = json.loads(self.r.get(key))
except:
return
return value
def lrange(self, key, start=0, end=-1):
try:
value = "".join(map(redis_decode, self.r.lrange(key, start, end) or []))
except:
return
return value
def lrange2(self, key, start=0, end=-1):
try:
return list(map(redis_decode, self.r.lrange(key, start, end) or []))
except:
return []
def llen(self, key):
try:
return self.r.llen(key) or 0
except:
return 0
def hget(self, key, field):
try:
return self.r.hget(key, field)
except Exception as e:
current_app.logger.warning("hget redis failed, %s" % str(e))
return
def hset(self, key, field, value):
try:
self.r.hset(key, field, value)
except Exception as e:
current_app.logger.warning("hset redis failed, %s" % str(e))
return
def expire(self, key, timeout):
try:
self.r.expire(key, timeout)
except Exception as e:
current_app.logger.warning("expire redis failed, %s" % str(e))
return
def redis_decode(x):
try:
return x.decode()
except Exception as e:
print(x, e)
try:
return x.decode("gb18030")
except:
return "decode failed"
class AESCrypto(object):
BLOCK_SIZE = 16 # Bytes
pad = lambda s: s + (AESCrypto.BLOCK_SIZE - len(s) % AESCrypto.BLOCK_SIZE) * \
chr(AESCrypto.BLOCK_SIZE - len(s) % AESCrypto.BLOCK_SIZE)
unpad = lambda s: s[:-ord(s[len(s) - 1:])]
iv = '0102030405060708'
@staticmethod
def key():
key = current_app.config.get("SECRET_KEY")[:16]
if len(key) < 16:
key = "{}{}".format(key, (16 - len(key) * "x"))
return key.encode('utf8')
@classmethod
def encrypt(cls, data):
data = cls.pad(data)
cipher = AES.new(cls.key(), AES.MODE_CBC, cls.iv.encode('utf8'))
return base64.b64encode(cipher.encrypt(data.encode('utf8'))).decode('utf8')
@classmethod
def decrypt(cls, data):
encode_bytes = base64.decodebytes(data.encode('utf8'))
cipher = AES.new(cls.key(), AES.MODE_CBC, cls.iv.encode('utf8'))
text_decrypted = cipher.decrypt(encode_bytes)
return cls.unpad(text_decrypted).decode('utf8')

2
cmdb-api/api/models/__init__.py
View File

@ -1,4 +1,4 @@
# -*- coding:utf-8 -*-
# -*- coding:utf-8 -*-
from .cmdb import *

180
cmdb-api/api/models/acl.py
View File

@ -13,6 +13,8 @@ from api.extensions import db
from api.lib.database import CRUDModel
from api.lib.database import Model
from api.lib.database import SoftDeleteMixin
from api.lib.perm.acl.const import ACL_QUEUE
from api.lib.perm.acl.const import OperateType
class App(Model):
@ -30,10 +32,13 @@ class UserQuery(BaseQuery):
def authenticate(self, login, password):
user = self.filter(db.or_(User.username == login,
User.email == login)).filter(User.deleted.is_(False)).first()
User.email == login)).filter(User.deleted.is_(False)).filter(User.block == 0).first()
if user:
current_app.logger.info(user)
authenticated = user.check_password(password)
if authenticated:
from api.tasks.acl import op_record
op_record.apply_async(args=(None, login, OperateType.LOGIN, ["ACL"]), queue=ACL_QUEUE)
else:
authenticated = False
@ -56,9 +61,11 @@ class UserQuery(BaseQuery):
ldap_conn.protocol_version = 3
ldap_conn.set_option(ldap.OPT_REFERRALS, 0)
if '@' in username:
email = username
who = '{0}@{1}'.format(username.split('@')[0], current_app.config.get('LDAP_DOMAIN'))
else:
who = '{0}@{1}'.format(username, current_app.config.get('LDAP_DOMAIN'))
email = who
username = username.split('@')[0]
user = self.get_by_username(username)
@ -71,7 +78,10 @@ class UserQuery(BaseQuery):
if not user:
from api.lib.perm.acl.user import UserCRUD
user = UserCRUD.add(username=username, email=who)
user = UserCRUD.add(username=username, email=email)
from api.tasks.acl import op_record
op_record.apply_async(args=(None, username, OperateType.LOGIN, ["ACL"]), queue=ACL_QUEUE)
return user, True
except ldap.INVALID_CREDENTIALS:
@ -80,28 +90,33 @@ class UserQuery(BaseQuery):
def search(self, key):
query = self.filter(db.or_(User.email == key,
User.nickname.ilike('%' + key + '%'),
User.username.ilike('%' + key + '%'))).filter(User.deleted.is_(False))
User.username.ilike('%' + key + '%')))
return query
def get_by_username(self, username):
user = self.filter(User.username == username).filter(User.deleted.is_(False)).first()
user = self.filter(User.username == username).first()
return user
def get_by_nickname(self, nickname):
user = self.filter(User.nickname == nickname).filter(User.deleted.is_(False)).first()
user = self.filter(User.nickname == nickname).first()
return user
def get_by_wxid(self, wx_id):
user = self.filter(User.wx_id == wx_id).first()
return user
def get(self, uid):
user = self.filter(User.uid == uid).filter(User.deleted.is_(False)).first()
user = self.filter(User.uid == uid).first()
return copy.deepcopy(user)
class User(CRUDModel, SoftDeleteMixin):
__tablename__ = 'users'
# __bind_key__ = "user"
__bind_key__ = "user"
query_class = UserQuery
uid = db.Column(db.Integer, primary_key=True, autoincrement=True)
@ -119,7 +134,9 @@ class User(CRUDModel, SoftDeleteMixin):
block = db.Column(db.Boolean, default=False)
has_logined = db.Column(db.Boolean, default=False)
wx_id = db.Column(db.String(32))
employee_id = db.Column(db.String(16), index=True)
avatar = db.Column(db.String(128))
# apps = db.Column(db.JSON)
def __str__(self):
return self.username
@ -145,16 +162,65 @@ class User(CRUDModel, SoftDeleteMixin):
def check_password(self, password):
if self.password is None:
return False
return self.password == password
return self.password == password or self.password == hashlib.md5(password.encode('utf-8')).hexdigest()
class RoleQuery(BaseQuery):
def _join(self, *args, **kwargs):
super(RoleQuery, self)._join(*args, **kwargs)
def authenticate(self, login, password):
role = self.filter(Role.name == login).first()
if role:
authenticated = role.check_password(password)
if authenticated:
from api.tasks.acl import op_record
op_record.apply_async(args=(None, login, OperateType.LOGIN, ["ACL"]), queue=ACL_QUEUE)
else:
authenticated = False
return role, authenticated
def authenticate_with_key(self, key, secret, args, path):
role = self.filter(Role.key == key).filter(Role.deleted.is_(False)).first()
if not role:
return None, False
if role and hashlib.sha1('{0}{1}{2}'.format(
path, role.secret, "".join(args)).encode("utf-8")).hexdigest() == secret:
authenticated = True
else:
authenticated = False
return role, authenticated
class Role(Model):
__tablename__ = "acl_roles"
query_class = RoleQuery
name = db.Column(db.Text, nullable=False)
name = db.Column(db.String(64), index=True, nullable=False)
is_app_admin = db.Column(db.Boolean, default=False)
app_id = db.Column(db.Integer, db.ForeignKey("acl_apps.id"))
uid = db.Column(db.Integer, db.ForeignKey("users.uid"))
uid = db.Column(db.Integer)
_password = db.Column("password", db.String(80))
key = db.Column(db.String(32))
secret = db.Column(db.String(32))
def _get_password(self):
return self._password
def _set_password(self, password):
if password:
self._password = hashlib.md5(password.encode('utf-8')).hexdigest()
password = db.synonym("_password", descriptor=property(_get_password, _set_password))
def check_password(self, password):
if self.password is None:
return False
return self.password == password or self.password == hashlib.md5(password.encode('utf-8')).hexdigest()
class RoleRelation(Model):
@ -162,6 +228,7 @@ class RoleRelation(Model):
parent_id = db.Column(db.Integer, db.ForeignKey('acl_roles.id'))
child_id = db.Column(db.Integer, db.ForeignKey('acl_roles.id'))
app_id = db.Column(db.Integer, db.ForeignKey('acl_apps.id'))
class ResourceType(Model):
@ -177,18 +244,24 @@ class ResourceGroup(Model):
name = db.Column(db.String(64), index=True, nullable=False)
resource_type_id = db.Column(db.Integer, db.ForeignKey("acl_resource_types.id"))
uid = db.Column(db.Integer, index=True)
app_id = db.Column(db.Integer, db.ForeignKey('acl_apps.id'))
resource_type = db.relationship("ResourceType", backref='acl_resource_groups.resource_type_id')
class Resource(Model):
__tablename__ = "acl_resources"
name = db.Column(db.String(128), nullable=False)
resource_type_id = db.Column(db.Integer, db.ForeignKey("acl_resource_types.id"))
uid = db.Column(db.Integer, index=True)
app_id = db.Column(db.Integer, db.ForeignKey("acl_apps.id"))
resource_type = db.relationship("ResourceType", backref='acl_resources.resource_type_id')
class ResourceGroupItems(Model):
__tablename__ = "acl_resource_group_items"
@ -196,6 +269,8 @@ class ResourceGroupItems(Model):
group_id = db.Column(db.Integer, db.ForeignKey('acl_resource_groups.id'), nullable=False)
resource_id = db.Column(db.Integer, db.ForeignKey('acl_resources.id'), nullable=False)
resource = db.relationship("Resource", backref='acl_resource_group_items.resource_id')
class Permission(Model):
__tablename__ = "acl_permissions"
@ -213,5 +288,90 @@ class RolePermission(Model):
resource_id = db.Column(db.Integer, db.ForeignKey('acl_resources.id'))
group_id = db.Column(db.Integer, db.ForeignKey('acl_resource_groups.id'))
perm_id = db.Column(db.Integer, db.ForeignKey('acl_permissions.id'))
app_id = db.Column(db.Integer, db.ForeignKey("acl_apps.id"))
perm = db.relationship("Permission", backref='acl_role_permissions.perm_id')
class Trigger(Model):
__tablename__ = "acl_triggers"
name = db.Column(db.String(128))
wildcard = db.Column(db.Text)
uid = db.Column(db.Text) # TODO
resource_type_id = db.Column(db.Integer, db.ForeignKey('acl_resource_types.id'))
roles = db.Column(db.Text) # TODO
permissions = db.Column(db.Text) # TODO
enabled = db.Column(db.Boolean, default=True)
app_id = db.Column(db.Integer, db.ForeignKey('acl_apps.id'))
class OperationRecord(Model):
__tablename__ = "acl_operation_records"
app = db.Column(db.String(32), index=True)
rolename = db.Column(db.String(32), index=True)
operate = db.Column(db.Enum(*OperateType.all()), nullable=False)
obj = db.Column(db.JSON)
class AuditRoleLog(Model):
__tablename__ = "acl_audit_role_logs"
app_id = db.Column(db.Integer, index=True)
operate_uid = db.Column(db.Integer, comment='操作人uid', index=True)
operate_type = db.Column(db.String(32), comment='操作类型', index=True)
scope = db.Column(db.String(16), comment='范围')
link_id = db.Column(db.Integer, comment='资源id', index=True)
origin = db.Column(db.JSON, default=dict(), comment='原始数据')
current = db.Column(db.JSON, default=dict(), comment='当前数据')
extra = db.Column(db.JSON, default=dict(), comment='其他内容')
source = db.Column(db.String(16), default='', comment='来源')
class AuditResourceLog(Model):
__tablename__ = "acl_audit_resource_logs"
app_id = db.Column(db.Integer, index=True)
operate_uid = db.Column(db.Integer, comment='操作人uid', index=True)
operate_type = db.Column(db.String(16), comment='操作类型', index=True)
scope = db.Column(db.String(16), comment='范围')
link_id = db.Column(db.Integer, comment='资源名', index=True)
origin = db.Column(db.JSON, default=dict(), comment='原始数据')
current = db.Column(db.JSON, default=dict(), comment='当前数据')
extra = db.Column(db.JSON, default=dict(), comment='权限名')
source = db.Column(db.String(16), default='', comment='来源')
class AuditPermissionLog(Model):
__tablename__ = "acl_audit_permission_logs"
app_id = db.Column(db.Integer, index=True)
operate_uid = db.Column(db.Integer, comment='操作人uid', index=True)
operate_type = db.Column(db.String(16), comment='操作类型', index=True)
rid = db.Column(db.Integer, comment='角色id', index=True)
resource_type_id = db.Column(db.Integer, comment='资源类型id', index=True)
resource_ids = db.Column(db.JSON, default=[], comment='资源')
group_ids = db.Column(db.JSON, default=[], comment='资源组')
permission_ids = db.Column(db.JSON, default=[], comment='权限')
source = db.Column(db.String(16), comment='来源')
class AuditTriggerLog(Model):
__tablename__ = "acl_audit_trigger_logs"
app_id = db.Column(db.Integer, index=True)
trigger_id = db.Column(db.Integer, comment='trigger', index=True)
operate_uid = db.Column(db.Integer, comment='操作人uid', index=True)
operate_type = db.Column(db.String(16), comment='操作类型', index=True)
origin = db.Column(db.JSON, default=dict(), comment='原始数据')
current = db.Column(db.JSON, default=dict(), comment='当前数据')
extra = db.Column(db.JSON, default=dict(), comment='权限名')
source = db.Column(db.String(16), default='', comment='来源')

197
cmdb-api/api/models/cmdb.py
View File

@ -3,11 +3,16 @@
import datetime
from sqlalchemy.dialects.mysql import DOUBLE
from api.extensions import db
from api.lib.cmdb.const import AutoDiscoveryType
from api.lib.cmdb.const import CIStatusEnum
from api.lib.cmdb.const import CITypeOperateType
from api.lib.cmdb.const import ConstraintEnum
from api.lib.cmdb.const import OperateType
from api.lib.cmdb.const import ValueTypeEnum
from api.lib.database import Model
from api.lib.database import Model, Model2
# template
@ -22,6 +27,7 @@ class CITypeGroup(Model):
__tablename__ = "c_ci_type_groups"
name = db.Column(db.String(32), nullable=False)
order = db.Column(db.Integer, default=0)
class CITypeGroupItem(Model):
@ -40,18 +46,22 @@ class CIType(Model):
unique_id = db.Column(db.Integer, db.ForeignKey("c_attributes.id"), nullable=False)
enabled = db.Column(db.Boolean, default=True, nullable=False)
is_attached = db.Column(db.Boolean, default=False, nullable=False)
icon_url = db.Column(db.String(256), default='', nullable=False)
icon = db.Column(db.Text)
order = db.Column(db.SmallInteger, default=0, nullable=False)
default_order_attr = db.Column(db.String(33))
unique_key = db.relationship("Attribute", backref="c_ci_types.unique_id")
uid = db.Column(db.Integer, index=True)
class CITypeRelation(Model):
__tablename__ = "c_ci_type_relations"
parent_id = db.Column(db.Integer, db.ForeignKey("c_ci_types.id"), nullable=False)
child_id = db.Column(db.Integer, db.ForeignKey("c_ci_types.id"), nullable=False)
parent_id = db.Column(db.Integer, db.ForeignKey("c_ci_types.id"), nullable=False) # source
child_id = db.Column(db.Integer, db.ForeignKey("c_ci_types.id"), nullable=False) # dst
relation_type_id = db.Column(db.Integer, db.ForeignKey("c_relation_types.id"), nullable=False)
constraint = db.Column(db.Enum(*ConstraintEnum.all()), default=ConstraintEnum.One2Many)
parent = db.relationship("CIType", primaryjoin="CIType.id==CITypeRelation.parent_id")
child = db.relationship("CIType", primaryjoin="CIType.id==CITypeRelation.child_id")
@ -73,6 +83,18 @@ class Attribute(Model):
is_password = db.Column(db.Boolean, default=False)
is_sortable = db.Column(db.Boolean, default=False)
default = db.Column(db.JSON) # {"default": None}
is_computed = db.Column(db.Boolean, default=False)
compute_expr = db.Column(db.Text)
compute_script = db.Column(db.Text)
choice_web_hook = db.Column(db.JSON)
uid = db.Column(db.Integer, index=True)
option = db.Column(db.JSON)
class CITypeAttribute(Model):
__tablename__ = "c_ci_type_attributes"
@ -102,6 +124,23 @@ class CITypeAttributeGroupItem(Model):
order = db.Column(db.SmallInteger, default=0)
class CITypeTrigger(Model):
# __tablename__ = "c_ci_type_triggers"
__tablename__ = "c_c_t_t"
type_id = db.Column(db.Integer, db.ForeignKey('c_ci_types.id'), nullable=False)
attr_id = db.Column(db.Integer, db.ForeignKey("c_attributes.id"), nullable=False)
notify = db.Column(db.JSON) # {subject: x, body: x, wx_to: [], mail_to: [], before_days: 0, notify_at: 08:00}
class CITypeUniqueConstraint(Model):
# __tablename__ = "c_ci_type_unique_constraints"
__tablename__ = "c_c_t_u_c"
type_id = db.Column(db.Integer, db.ForeignKey('c_ci_types.id'), nullable=False)
attr_ids = db.Column(db.JSON) # [attr_id, ]
# instance
class CI(Model):
@ -110,6 +149,7 @@ class CI(Model):
type_id = db.Column(db.Integer, db.ForeignKey("c_ci_types.id"), nullable=False)
status = db.Column(db.Enum(*CIStatusEnum.all(), name="status"))
heartbeat = db.Column(db.DateTime, default=lambda: datetime.datetime.now())
is_auto_discovery = db.Column('a', db.Boolean, default=False)
ci_type = db.relationship("CIType", backref="c_cis.type_id")
@ -132,6 +172,7 @@ class IntegerChoice(Model):
attr_id = db.Column(db.Integer, db.ForeignKey('c_attributes.id'), nullable=False)
value = db.Column(db.Integer, nullable=False)
option = db.Column(db.JSON)
attr = db.relationship("Attribute", backref="c_choice_integers.attr_id")
@ -140,7 +181,8 @@ class FloatChoice(Model):
__tablename__ = 'c_choice_floats'
attr_id = db.Column(db.Integer, db.ForeignKey('c_attributes.id'), nullable=False)
value = db.Column(db.Float, nullable=False)
value = db.Column(DOUBLE, nullable=False)
option = db.Column(db.JSON)
attr = db.relationship("Attribute", backref="c_choice_floats.attr_id")
@ -150,6 +192,7 @@ class TextChoice(Model):
attr_id = db.Column(db.Integer, db.ForeignKey('c_attributes.id'), nullable=False)
value = db.Column(db.Text, nullable=False)
option = db.Column(db.JSON)
attr = db.relationship("Attribute", backref="c_choice_texts.attr_id")
@ -172,7 +215,7 @@ class CIIndexValueFloat(Model):
ci_id = db.Column(db.Integer, db.ForeignKey('c_cis.id'), nullable=False)
attr_id = db.Column(db.Integer, db.ForeignKey('c_attributes.id'), nullable=False)
value = db.Column(db.Float, nullable=False)
value = db.Column(DOUBLE, nullable=False)
ci = db.relationship("CI", backref="c_value_index_floats.ci_id")
attr = db.relationship("Attribute", backref="c_value_index_floats.attr_id")
@ -222,7 +265,7 @@ class CIValueFloat(Model):
ci_id = db.Column(db.Integer, db.ForeignKey('c_cis.id'), nullable=False)
attr_id = db.Column(db.Integer, db.ForeignKey('c_attributes.id'), nullable=False)
value = db.Column(db.Float, nullable=False)
value = db.Column(DOUBLE, nullable=False)
ci = db.relationship("CI", backref="c_value_floats.ci_id")
attr = db.relationship("Attribute", backref="c_value_floats.attr_id")
@ -262,7 +305,7 @@ class CIValueJson(Model):
# history
class OperationRecord(Model):
class OperationRecord(Model2):
__tablename__ = "c_records"
uid = db.Column(db.Integer, index=True, nullable=False)
@ -270,6 +313,8 @@ class OperationRecord(Model):
ticket_id = db.Column(db.String(32), nullable=True)
reason = db.Column(db.Text)
type_id = db.Column(db.Integer, index=True)
class AttributeHistory(Model):
__tablename__ = "c_attribute_histories"
@ -293,29 +338,151 @@ class CIRelationHistory(Model):
relation_id = db.Column(db.Integer, nullable=False)
class CITypeHistory(Model):
__tablename__ = "c_ci_type_histories"
operate_type = db.Column(db.Enum(*CITypeOperateType.all(), name="operate_type"))
type_id = db.Column(db.Integer, index=True, nullable=False)
attr_id = db.Column(db.Integer)
trigger_id = db.Column(db.Integer)
unique_constraint_id = db.Column(db.Integer)
uid = db.Column(db.Integer, index=True)
change = db.Column(db.JSON)
# preference
class PreferenceShowAttributes(Model):
__tablename__ = "c_preference_show_attributes"
# __tablename__ = "c_preference_show_attributes"
__tablename__ = "c_psa"
uid = db.Column(db.Integer, index=True, nullable=False)
type_id = db.Column(db.Integer, db.ForeignKey("c_ci_types.id"), nullable=False)
attr_id = db.Column(db.Integer, db.ForeignKey("c_attributes.id"))
order = db.Column(db.SmallInteger, default=0)
is_fixed = db.Column(db.Boolean, default=False)
ci_type = db.relationship("CIType", backref="c_preference_show_attributes.type_id")
attr = db.relationship("Attribute", backref="c_preference_show_attributes.attr_id")
ci_type = db.relationship("CIType", backref="c_psa.type_id")
attr = db.relationship("Attribute", backref="c_psa.attr_id")
class PreferenceTreeView(Model):
__tablename__ = "c_preference_tree_views"
# __tablename__ = "c_preference_tree_views"
__tablename__ = "c_ptv"
uid = db.Column(db.Integer, index=True, nullable=False)
type_id = db.Column(db.Integer, db.ForeignKey("c_ci_types.id"), nullable=False)
levels = db.Column(db.Text) # TODO: JSON
levels = db.Column(db.JSON)
class PreferenceRelationView(Model):
__tablename__ = "c_preference_relation_views"
# __tablename__ = "c_preference_relation_views"
__tablename__ = "c_prv"
uid = db.Column(db.Integer, index=True, nullable=False)
name = db.Column(db.String(64), index=True, nullable=False)
cr_ids = db.Column(db.TEXT) # [{parent_id: x, child_id: y}] TODO: JSON
cr_ids = db.Column(db.JSON) # [{parent_id: x, child_id: y}]
is_public = db.Column(db.Boolean, default=False)
class PreferenceSearchOption(Model):
__tablename__ = "c_pso"
name = db.Column(db.String(64))
prv_id = db.Column(db.Integer, db.ForeignKey("c_prv.id"))
ptv_id = db.Column(db.Integer, db.ForeignKey("c_ptv.id"))
type_id = db.Column(db.Integer, db.ForeignKey("c_ci_types.id"))
uid = db.Column(db.Integer, index=True)
option = db.Column(db.JSON)
# custom
class CustomDashboard(Model):
__tablename__ = "c_c_d"
name = db.Column(db.String(64))
category = db.Column(db.SmallInteger) # 0: 总数统计, 1: 字段值统计, 2: 关系统计
enabled = db.Column(db.Boolean, default=False)
order = db.Column(db.Integer, default=0)
type_id = db.Column(db.Integer, db.ForeignKey('c_ci_types.id'))
attr_id = db.Column(db.Integer, db.ForeignKey('c_attributes.id'))
level = db.Column(db.Integer)
options = db.Column(db.JSON)
class SystemConfig(Model):
__tablename__ = "c_sc"
name = db.Column(db.String(64), index=True)
option = db.Column(db.JSON)
# auto discovery
class AutoDiscoveryRule(Model):
__tablename__ = "c_ad_rules"
name = db.Column(db.String(32))
type = db.Column(db.Enum(*AutoDiscoveryType.all()), index=True)
is_inner = db.Column(db.Boolean, default=False, index=True)
owner = db.Column(db.Integer, index=True)
option = db.Column(db.JSON) # layout
attributes = db.Column(db.JSON)
is_plugin = db.Column(db.Boolean, default=False)
plugin_script = db.Column(db.Text)
unique_key = db.Column(db.String(64))
class AutoDiscoveryCIType(Model):
__tablename__ = "c_ad_ci_types"
type_id = db.Column(db.Integer, db.ForeignKey('c_ci_types.id'))
adr_id = db.Column(db.Integer, db.ForeignKey('c_ad_ci_types.id'))
attributes = db.Column(db.JSON) # {ad_key: cmdb_key}
relation = db.Column(db.JSON) # [{ad_key: {type_id: x, attr_id: x}}]
auto_accept = db.Column(db.Boolean, default=False)
agent_id = db.Column(db.String(8), index=True)
query_expr = db.Column(db.Text)
interval = db.Column(db.Integer) # seconds
cron = db.Column(db.String(128))
extra_option = db.Column(db.JSON)
uid = db.Column(db.Integer, index=True)
class AutoDiscoveryCI(Model):
__tablename__ = "c_ad_ci"
type_id = db.Column(db.Integer, db.ForeignKey('c_ci_types.id'))
adt_id = db.Column(db.Integer, db.ForeignKey('c_ad_ci_types.id'))
unique_value = db.Column(db.String(128), index=True)
instance = db.Column(db.JSON)
ci_id = db.Column(db.Integer, index=True)
is_accept = db.Column(db.Boolean, default=False)
accept_by = db.Column(db.String(64), index=True)
accept_time = db.Column(db.DateTime)
class CIFilterPerms(Model):
__tablename__ = "c_ci_filter_perms"
name = db.Column(db.String(64), index=True)
type_id = db.Column(db.Integer, db.ForeignKey('c_ci_types.id'))
ci_filter = db.Column(db.Text)
attr_filter = db.Column(db.Text)
rid = db.Column(db.Integer, index=True)

87
cmdb-api/api/models/common_setting.py Normal file
View File

@ -0,0 +1,87 @@
from api.extensions import db
from api.lib.database import Model, TimestampMixin, SoftDeleteMixin, CRUDMixin
class ModelWithoutPK(db.Model, TimestampMixin, SoftDeleteMixin, CRUDMixin):
__table_args__ = {"extend_existing": True}
__abstract__ = True
class Department(ModelWithoutPK):
__tablename__ = 'common_department'
department_id = db.Column(db.Integer, primary_key=True, autoincrement=True)
department_name = db.Column(db.VARCHAR(255), default='', comment='部门名称')
department_director_id = db.Column(
db.Integer, default=0, comment='部门负责人ID')
department_parent_id = db.Column(db.Integer, default=1, comment='上级部门ID')
sort_value = db.Column(db.Integer, default=0, comment='排序值')
acl_rid = db.Column(db.Integer, comment='ACL中rid', default=0)
class Employee(ModelWithoutPK):
__tablename__ = 'common_employee'
employee_id = db.Column(db.Integer, primary_key=True, autoincrement=True)
email = db.Column(db.VARCHAR(255), default='', comment='邮箱')
username = db.Column(db.VARCHAR(255), default='', comment='用户名')
nickname = db.Column(db.VARCHAR(255), default='', comment='姓名')
sex = db.Column(db.VARCHAR(64), default='', comment='性别')
position_name = db.Column(db.VARCHAR(255), default='', comment='职位名称')
mobile = db.Column(db.VARCHAR(255), default='', comment='电话号码')
avatar = db.Column(db.VARCHAR(255), default='', comment='头像')
direct_supervisor_id = db.Column(db.Integer, default=0, comment='直接上级ID')
department_id = db.Column(db.Integer,
db.ForeignKey('common_department.department_id'),
comment='部门ID',
)
acl_uid = db.Column(db.Integer, comment='ACL中uid', default=0)
acl_rid = db.Column(db.Integer, comment='ACL中rid', default=0)
acl_virtual_rid = db.Column(db.Integer, comment='ACL中虚拟角色rid', default=0)
last_login = db.Column(db.TIMESTAMP, nullable=True, comment='上次登录时间')
block = db.Column(db.Integer, comment='锁定状态', default=0)
_department = db.relationship(
'Department', backref='common_employee.department_id',
lazy='joined'
)
class EmployeeInfo(Model):
"""
员工信息
"""
__tablename__ = 'common_employee_info'
info = db.Column(db.JSON, default={}, comment='员工信息')
employee_id = db.Column(db.Integer, db.ForeignKey(
'common_employee.employee_id'), comment='员工ID')
employee = db.relationship(
'Employee', backref='common_employee.employee_id', lazy='joined')
class CompanyInfo(Model):
__tablename__ = "common_company_info_json"
info = db.Column(db.JSON)
class InternalMessage(Model):
"""
内部消息
"""
__tablename__ = "common_internal_message"
title = db.Column(db.VARCHAR(255), nullable=True, comment='标题')
content = db.Column(db.TEXT, nullable=True, comment='内容')
path = db.Column(db.VARCHAR(255), nullable=True, comment='跳转路径')
is_read = db.Column(db.Boolean, default=False, comment='是否已读')
app_name = db.Column(db.VARCHAR(128), nullable=False, comment='应用名称')
category = db.Column(db.VARCHAR(128), nullable=False, comment='分类')
message_data = db.Column(db.JSON, nullable=True, comment='数据')
employee_id = db.Column(db.Integer, db.ForeignKey('common_employee.employee_id'), comment='ID')

5
cmdb-api/api/resource.py
View File

@ -6,6 +6,7 @@ from inspect import getmembers, isclass
import six
from flask import jsonify
from flask import send_file
from flask_restful import Resource
from api.lib.perm.auth import auth_required
@ -21,6 +22,10 @@ class APIView(Resource):
def jsonify(*args, **kwargs):
return jsonify(*args, **kwargs)
@staticmethod
def send_file(*args, **kwargs):
return send_file(*args, **kwargs)
API_PACKAGE = "api"

192
cmdb-api/api/tasks/acl.py
View File

@ -1,16 +1,200 @@
# -*- coding:utf-8 -*-
import json
import re
from celery_once import QueueOnce
from flask import current_app
from werkzeug.exceptions import BadRequest, NotFound
from api.extensions import celery
from api.extensions import db
from api.lib.perm.acl.cache import AppCache
from api.lib.perm.acl.cache import RoleCache
from api.lib.perm.acl.cache import RoleRelationCache
from api.lib.perm.acl.cache import UserCache
from api.lib.perm.acl.const import ACL_QUEUE
from api.lib.perm.acl.record import OperateRecordCRUD
from api.lib.perm.acl.audit import AuditCRUD, AuditOperateType, AuditOperateSource
from api.models.acl import Resource
from api.models.acl import Role
from api.models.acl import Trigger
@celery.task(name="acl.role_rebuild", queue=ACL_QUEUE)
def role_rebuild(rids):
@celery.task(base=QueueOnce,
name="acl.role_rebuild",
queue=ACL_QUEUE,
once={"graceful": True, "unlock_before_run": True})
def role_rebuild(rids, app_id):
rids = rids if isinstance(rids, list) else [rids]
for rid in rids:
RoleRelationCache.rebuild(rid)
RoleRelationCache.rebuild(rid, app_id)
current_app.logger.info("Role {0} rebuild..........".format(rids))
current_app.logger.info("Role {0} App {1} rebuild..........".format(rids, app_id))
@celery.task(name="acl.update_resource_to_build_role", queue=ACL_QUEUE)
def update_resource_to_build_role(resource_id, app_id, group_id=None):
rids = [i.id for i in Role.get_by(__func_isnot__key_uid=None, fl='id', to_dict=False)]
rids += [i.id for i in Role.get_by(app_id=app_id, fl='id', to_dict=False)]
rids += [i.id for i in Role.get_by(__func_is___key_uid=None, __func_is___key_app_id=None, fl='id', to_dict=False)]
current_app.logger.info(rids)
for rid in rids:
if resource_id and resource_id in RoleRelationCache.get_resources(rid, app_id).get('id2perms', {}):
RoleRelationCache.rebuild2(rid, app_id)
if group_id and group_id in RoleRelationCache.get_resources(rid, app_id).get('group2perms', {}):
RoleRelationCache.rebuild2(rid, app_id)
@celery.task(name="acl.apply_trigger", queue=ACL_QUEUE)
def apply_trigger(_id, resource_id=None, operator_uid=None):
db.session.remove()
from api.lib.perm.acl.permission import PermissionCRUD
trigger = Trigger.get_by_id(_id)
if trigger is None:
return
uid = json.loads(trigger.uid or '[]')
if resource_id is None:
wildcard = (trigger.wildcard or '')
if wildcard and uid:
query = Resource.get_by(__func_in___key_uid=uid,
app_id=trigger.app_id,
resource_type_id=trigger.resource_type_id,
fl=['id', 'app_id'],
only_query=True)
try:
re.compile(wildcard)
resources = query.filter(Resource.name.op('regexp')(wildcard)).all()
except:
resources = query.filter(Resource.name.ilike(wildcard.replace('*', '%'))).all()
elif wildcard:
query = Resource.get_by(app_id=trigger.app_id,
resource_type_id=trigger.resource_type_id,
only_query=True)
try:
re.compile(wildcard)
resources = query.filter(Resource.name.op('regexp')(wildcard)).all()
except:
resources = query.filter(Resource.name.ilike(wildcard.replace('*', '%'))).all()
elif uid:
resources = Resource.get_by(__func_in___key_uid=uid,
app_id=trigger.app_id,
resource_type_id=trigger.resource_type_id,
to_dict=False)
else:
resources = []
else:
resources = [Resource.get_by_id(resource_id)]
perms = json.loads(trigger.permissions)
roles = json.loads(trigger.roles)
for resource in resources:
for rid in roles:
try:
PermissionCRUD.grant(rid, perms, resource.id, rebuild=False, source=AuditOperateSource.trigger)
except (NotFound, BadRequest):
pass
AuditCRUD.add_trigger_log(trigger.app_id, trigger.id, AuditOperateType.trigger_apply, {}, trigger.to_dict(),
{'uid': uid,
'resource_ids': [r.id for r in resources],
'perms': perms,
'rids': roles},
uid=operator_uid, source=AuditOperateSource.trigger)
if resources:
role_rebuild(roles, resources[0].app_id)
@celery.task(name="acl.cancel_trigger", queue=ACL_QUEUE)
def cancel_trigger(_id, resource_id=None, operator_uid=None):
db.session.remove()
from api.lib.perm.acl.permission import PermissionCRUD
trigger = Trigger.get_by_id(_id)
if trigger is None:
return
uid = json.loads(trigger.uid or '[]')
if resource_id is None:
wildcard = (trigger.wildcard or '')
if wildcard and uid:
query = Resource.get_by(__func_in___key_uid=uid,
app_id=trigger.app_id,
resource_type_id=trigger.resource_type_id,
fl=['id', 'app_id'],
only_query=True)
try:
re.compile(wildcard)
resources = query.filter(Resource.name.op('regexp')(wildcard)).all()
except:
resources = query.filter(Resource.name.ilike(wildcard.replace('*', '%'))).all()
elif wildcard:
query = Resource.get_by(app_id=trigger.app_id,
resource_type_id=trigger.resource_type_id,
only_query=True)
try:
re.compile(wildcard)
resources = query.filter(Resource.name.op('regexp')(wildcard)).all()
except:
resources = query.filter(Resource.name.ilike(wildcard.replace('*', '%'))).all()
elif uid:
resources = Resource.get_by(__func_in___key_uid=uid,
app_id=trigger.app_id,
resource_type_id=trigger.resource_type_id,
to_dict=False)
else:
resources = []
else:
resources = [Resource.get_by_id(resource_id)]
perms = json.loads(trigger.permissions)
roles = json.loads(trigger.roles)
for resource in resources:
if not resource:
continue
for rid in roles:
try:
PermissionCRUD.revoke(rid, perms, resource.id, rebuild=False, source=AuditOperateSource.trigger)
except (NotFound, BadRequest):
pass
AuditCRUD.add_trigger_log(trigger.app_id, trigger.id, AuditOperateType.trigger_cancel, {}, trigger.to_dict(),
{'uid': uid,
'resource_ids': [r.id for r in resources if r],
'perms': perms,
'rids': roles},
uid=operator_uid, source=AuditOperateSource.trigger)
if resources:
role_rebuild(roles, resources[0].app_id)
@celery.task(name="acl.op_record", queue=ACL_QUEUE)
def op_record(app, rolename, operate_type, obj):
if isinstance(app, int):
app = AppCache.get(app)
app = app and app.name
if isinstance(rolename, int):
u = UserCache.get(rolename)
if u:
rolename = u.username
if not u:
r = RoleCache.get(rolename)
if r:
rolename = r.name
OperateRecordCRUD.add(app, rolename, operate_type, obj)

96
cmdb-api/api/tasks/cmdb.py
View File

@ -4,6 +4,8 @@
import json
import time
import jinja2
import requests
from flask import current_app
import api.lib.cmdb.ci
@ -15,24 +17,44 @@ from api.lib.cmdb.cache import CITypeAttributesCache
from api.lib.cmdb.const import CMDB_QUEUE
from api.lib.cmdb.const import REDIS_PREFIX_CI
from api.lib.cmdb.const import REDIS_PREFIX_CI_RELATION
from api.lib.mail import send_mail
from api.lib.utils import Lock
from api.models.cmdb import CIRelation
@celery.task(name="cmdb.ci_cache", queue=CMDB_QUEUE)
def ci_cache(ci_id):
time.sleep(0.01)
db.session.close()
db.session.remove()
m = api.lib.cmdb.ci.CIManager()
ci = m.get_ci_by_id_from_db(ci_id, need_children=False, use_master=False)
ci_dict = m.get_ci_by_id_from_db(ci_id, need_children=False, use_master=False)
if current_app.config.get("USE_ES"):
es.create_or_update(ci_id, ci)
es.create_or_update(ci_id, ci_dict)
else:
rd.create_or_update({ci_id: json.dumps(ci)}, REDIS_PREFIX_CI)
rd.create_or_update({ci_id: json.dumps(ci_dict)}, REDIS_PREFIX_CI)
current_app.logger.info("{0} flush..........".format(ci_id))
@celery.task(name="cmdb.batch_ci_cache", queue=CMDB_QUEUE)
def batch_ci_cache(ci_ids):
time.sleep(1)
db.session.remove()
for ci_id in ci_ids:
m = api.lib.cmdb.ci.CIManager()
ci_dict = m.get_ci_by_id_from_db(ci_id, need_children=False, use_master=False)
if current_app.config.get("USE_ES"):
es.create_or_update(ci_id, ci_dict)
else:
rd.create_or_update({ci_id: json.dumps(ci_dict)}, REDIS_PREFIX_CI)
current_app.logger.info("{0} flush..........".format(ci_id))
@celery.task(name="cmdb.ci_delete", queue=CMDB_QUEUE)
def ci_delete(ci_id):
current_app.logger.info(ci_id)
@ -47,29 +69,31 @@ def ci_delete(ci_id):
@celery.task(name="cmdb.ci_relation_cache", queue=CMDB_QUEUE)
def ci_relation_cache(parent_id, child_id):
db.session.close()
db.session.remove()
children = rd.get([parent_id], REDIS_PREFIX_CI_RELATION)[0]
children = json.loads(children) if children is not None else {}
with Lock("CIRelation_{}".format(parent_id)):
children = rd.get([parent_id], REDIS_PREFIX_CI_RELATION)[0]
children = json.loads(children) if children is not None else {}
cr = CIRelation.get_by(first_ci_id=parent_id, second_ci_id=child_id, first=True, to_dict=False)
if str(child_id) not in children:
children[str(child_id)] = cr.second_ci.type_id
cr = CIRelation.get_by(first_ci_id=parent_id, second_ci_id=child_id, first=True, to_dict=False)
if str(child_id) not in children:
children[str(child_id)] = cr.second_ci.type_id
rd.create_or_update({parent_id: json.dumps(children)}, REDIS_PREFIX_CI_RELATION)
rd.create_or_update({parent_id: json.dumps(children)}, REDIS_PREFIX_CI_RELATION)
current_app.logger.info("ADD ci relation cache: {0} -> {1}".format(parent_id, child_id))
@celery.task(name="cmdb.ci_relation_delete", queue=CMDB_QUEUE)
def ci_relation_delete(parent_id, child_id):
children = rd.get([parent_id], REDIS_PREFIX_CI_RELATION)[0]
children = json.loads(children) if children is not None else {}
with Lock("CIRelation_{}".format(parent_id)):
children = rd.get([parent_id], REDIS_PREFIX_CI_RELATION)[0]
children = json.loads(children) if children is not None else {}
if str(child_id) in children:
children.pop(str(child_id))
if str(child_id) in children:
children.pop(str(child_id))
rd.create_or_update({parent_id: json.dumps(children)}, REDIS_PREFIX_CI_RELATION)
rd.create_or_update({parent_id: json.dumps(children)}, REDIS_PREFIX_CI_RELATION)
current_app.logger.info("DELETE ci relation cache: {0} -> {1}".format(parent_id, child_id))
@ -92,3 +116,43 @@ def ci_type_attribute_order_rebuild(type_id):
id2attr.get(_attr['id']).update(order=order)
order += 1
@celery.task(name='cmdb.trigger_notify', queue=CMDB_QUEUE)
def trigger_notify(notify, ci_id):
from api.lib.perm.acl.cache import UserCache
def _wrap_mail(mail_to):
if "@" not in mail_to:
user = UserCache.get(mail_to)
if user:
return user.email
return mail_to
db.session.remove()
m = api.lib.cmdb.ci.CIManager()
ci_dict = m.get_ci_by_id_from_db(ci_id, need_children=False, use_master=False)
subject = jinja2.Template(notify.get('subject') or "").render(ci_dict)
body = jinja2.Template(notify.get('body') or "").render(ci_dict)
if notify.get('wx_to'):
to_user = jinja2.Template('|'.join(notify['wx_to'])).render(ci_dict)
url = current_app.config.get("WX_URI")
data = {"to_user": to_user, "content": subject}
try:
requests.post(url, data=data)
except Exception as e:
current_app.logger.error(str(e))
if notify.get('mail_to'):
try:
if len(subject) > 700:
subject = subject[:600] + "..." + subject[-100:]
send_mail("", [_wrap_mail(jinja2.Template(i).render(ci_dict))
for i in notify['mail_to'] if i], subject, body)
except Exception as e:
current_app.logger.error("Send mail failed: {0}".format(str(e)))

84
cmdb-api/api/tasks/common_setting.py Normal file
View File

@ -0,0 +1,84 @@
# -*- coding:utf-8 -*-
import requests
from flask import current_app
from api.extensions import celery
from api.extensions import db
from api.lib.common_setting.acl import ACLManager
from api.lib.common_setting.const import COMMON_SETTING_QUEUE
from api.lib.common_setting.resp_format import ErrFormat
from api.models.common_setting import Department
@celery.task(name="common_setting.edit_employee_department_in_acl", queue=COMMON_SETTING_QUEUE)
def edit_employee_department_in_acl(e_list, new_d_id, op_uid):
"""
在 ACL 员工更换部门
:param e_list: 员工列表 {acl_rid: 11, department_id: 22}
:param new_d_id: 新部门 ID
:param op_uid: 操作人 ID
在老部门中删除员工
在新部门中添加员工
"""
db.session.remove()
result = []
new_department = Department.get_by(
first=True, department_id=new_d_id, to_dict=False)
if not new_department:
result.append(ErrFormat.new_department_is_none)
return result
acl = ACLManager('acl', str(op_uid))
role_map = {role['name']: role['id'] for role in acl.get_all_roles()}
new_d_rid_in_acl = role_map.get(new_department.department_name, 0)
if new_d_rid_in_acl == 0:
return
if new_d_rid_in_acl != new_department.acl_rid:
new_department.update(
acl_rid=new_d_rid_in_acl
)
new_department_acl_rid = new_department.acl_rid if new_d_rid_in_acl == new_department.acl_rid else new_d_rid_in_acl
for employee in e_list:
# 根据 部门ID获取部门 acl_rid
old_department = Department.get_by(
first=True, department_id=employee.get('department_id'), to_dict=False)
if not old_department:
continue
employee_acl_rid = employee.get('e_acl_rid')
if employee_acl_rid == 0:
result.append(ErrFormat.employee_acl_rid_is_zero)
continue
old_d_rid_in_acl = role_map.get(old_department.department_name, 0)
if old_d_rid_in_acl == 0:
return
if old_d_rid_in_acl != old_department.acl_rid:
old_department.update(
acl_rid=old_d_rid_in_acl
)
d_acl_rid = old_department.acl_rid if old_d_rid_in_acl == old_department.acl_rid else old_d_rid_in_acl
# 在老部门中删除员工
payload = {
'app_id': 'acl',
'parent_id': d_acl_rid,
}
try:
acl.remove_user_from_role(employee_acl_rid, payload)
except Exception as e:
result.append(ErrFormat.acl_remove_user_from_role_failed.format(str(e)))
# 在新部门中添加员工
payload = {
'app_id': 'acl',
'child_ids': [employee_acl_rid],
}
try:
acl.add_user_to_role(new_department_acl_rid, payload)
except Exception as e:
result.append(ErrFormat.acl_add_user_to_role_failed.format(str(e)))
return result

10
cmdb-api/api/tasks/test.py
View File

@ -1,10 +0,0 @@
# -*- coding:utf-8 -*-
from flask import current_app
from api.extensions import celery
@celery.task(queue="ticket_web")
def test_task():
current_app.logger.info("test task.............................")

27
cmdb-api/api/views/__init__.py
View File

@ -1,29 +1,2 @@
# -*- coding:utf-8 -*-
import os
from flask import Blueprint
from flask_restful import Api
from api.resource import register_resources
from api.views.account import LoginView, LogoutView
HERE = os.path.abspath(os.path.dirname(__file__))
# account
blueprint_account = Blueprint('account_api', __name__, url_prefix='/api')
account_rest = Api(blueprint_account)
account_rest.add_resource(LoginView, LoginView.url_prefix)
account_rest.add_resource(LogoutView, LogoutView.url_prefix)
# cmdb
blueprint_cmdb_v01 = Blueprint('cmdb_api_v01', __name__, url_prefix='/api/v0.1')
rest = Api(blueprint_cmdb_v01)
register_resources(os.path.join(HERE, "cmdb"), rest)
# acl
blueprint_acl_v1 = Blueprint('acl_api_v1', __name__, url_prefix='/api/v1/acl')
rest = Api(blueprint_acl_v1)
register_resources(os.path.join(HERE, "acl"), rest)

50
cmdb-api/api/views/account.py
View File

@ -2,19 +2,17 @@
import datetime
import six
import jwt
from flask import abort
from flask import current_app
from flask import request
from flask import session
from flask_login import login_user, logout_user
from api.lib.decorator import args_required
from api.lib.perm.acl.cache import User
from api.lib.perm.auth import auth_abandoned
from api.models.acl import User, Role
from api.resource import APIView
from api.lib.perm.acl.role import RoleRelationCRUD
from api.lib.perm.acl.cache import RoleCache
class LoginView(APIView):
@ -26,14 +24,9 @@ class LoginView(APIView):
def post(self):
username = request.values.get("username") or request.values.get("email")
password = request.values.get("password")
if current_app.config.get('AUTH_WITH_LDAP'):
user, authenticated = User.query.authenticate_with_ldap(username, password)
else:
user, authenticated = User.query.authenticate(username, password)
if not user:
return abort(403, "User <{0}> does not exist".format(username))
user, authenticated = User.query.authenticate(username, password)
if not authenticated:
return abort(403, "invalid username or password")
return abort(401, "invalid username or password")
login_user(user)
@ -43,19 +36,30 @@ class LoginView(APIView):
'exp': datetime.datetime.now() + datetime.timedelta(minutes=24 * 60 * 7)},
current_app.config['SECRET_KEY'])
role = Role.get_by(uid=user.uid, first=True, to_dict=False)
if role:
parent_ids = RoleRelationCRUD.recursive_parent_ids(role.id)
parent_roles = [RoleCache.get(i).name for i in parent_ids]
else:
parent_roles = []
session["acl"] = dict(uid=user.uid,
avatar=user.avatar,
userName=user.username,
nickName=user.nickname,
parentRoles=parent_roles)
return self.jsonify(token=token.decode() if six.PY2 else token, username=username)
return self.jsonify(token=token.decode())
class AuthWithKeyView(APIView):
url_prefix = "/auth_with_key"
@args_required("key")
@args_required("secret")
@args_required("path")
@auth_abandoned
def post(self):
key = request.values.get('key')
secret = request.values.get('secret')
path = six.moves.urllib.parse.urlparse(request.values.get('path')).path
payload = request.values.get('payload') or {}
payload.pop('_key', None)
payload.pop('_secret', None)
req_args = [str(payload[k]) for k in sorted(payload.keys())]
user, authenticated = User.query.authenticate_with_key(key, secret, req_args, path)
return self.jsonify(user=user.to_dict() if user else {},
authenticated=authenticated)
class LogoutView(APIView):

78
cmdb-api/api/views/acl/app.py Normal file
View File

@ -0,0 +1,78 @@
# -*- coding:utf-8 -*-
from flask import abort
from flask import request
from api.lib.decorator import args_required
from api.lib.decorator import args_validate
from api.lib.perm.acl.acl import is_app_admin
from api.lib.perm.acl.app import AppCRUD
from api.lib.perm.acl.resp_format import ErrFormat
from api.lib.perm.auth import auth_abandoned
from api.lib.utils import get_page
from api.lib.utils import get_page_size
from api.resource import APIView
class AppView(APIView):
url_prefix = ('/apps', '/apps/<int:_id>')
def get(self, _id=None):
if _id is not None:
if not is_app_admin('acl'):
return abort(403, ErrFormat.no_permission)
app = AppCRUD.get(_id)
app = app and app.to_dict() or {}
return self.jsonify(**app)
page = get_page(request.values.get('page', 1))
page_size = get_page_size(request.values.get('page_size'))
q = request.values.get('q')
numfound, res = AppCRUD.search(q, page, page_size)
res = [i.to_dict() for i in res]
for i in res:
i.pop('app_id', None)
i.pop('secret_key', None)
return self.jsonify(page=page,
page_size=page_size,
numfound=numfound,
total=len(res),
apps=res)
@args_required('name')
@args_validate(AppCRUD.cls)
def post(self):
name = request.values.get('name')
description = request.values.get('description')
app = AppCRUD.add(name, description)
return self.jsonify(app.to_dict())
@args_validate(AppCRUD.cls)
def put(self, _id):
app = AppCRUD.update(_id, **request.values)
return self.jsonify(app.to_dict())
def delete(self, _id):
AppCRUD.delete(_id)
return self.jsonify(id=_id)
class AppAccessTokenView(APIView):
url_prefix = '/apps/token'
@args_required('app_id')
@args_required('secret_key')
@auth_abandoned
def post(self):
token = AppCRUD.gen_token(request.values.get('app_id'), request.values.get('secret_key'))
return self.jsonify(token=token)

39
cmdb-api/api/views/acl/audit.py Normal file
View File

@ -0,0 +1,39 @@
# -*- coding:utf-8 -*-
from flask import request, abort
from api.lib.perm.acl.audit import AuditCRUD
from api.lib.utils import get_page
from api.lib.utils import get_page_size
from api.resource import APIView
class AuditLogView(APIView):
url_prefix = ("/audit_log/<string:name>",)
def get(self, name):
page = get_page(request.values.get("page", 1))
page_size = get_page_size(request.values.get("page_size"))
app_id = request.values.get('app_id')
q = request.values.get('q')
start = request.values.get('start')
end = request.values.get('end')
func_map = {
'permission': AuditCRUD.search_permission,
'role': AuditCRUD.search_role,
'trigger': AuditCRUD.search_trigger,
'resource': AuditCRUD.search_resource,
}
if name not in func_map:
abort(400, f'wrong {name}, please use {func_map.keys()}')
_func = func_map[name]
data = _func(app_id, q, page, page_size, start, end)
return self.jsonify(
page=page,
page_size=page_size,
**data,
)

179
cmdb-api/api/views/acl/login.py Normal file
View File

@ -0,0 +1,179 @@
# -*- coding:utf-8 -*-
import datetime
import jwt
import six
from flask import abort
from flask import current_app
from flask import request
from flask import session
from flask_login import login_user, logout_user
from api.lib.decorator import args_required
from api.lib.decorator import args_validate
from api.lib.perm.acl.acl import ACLManager
from api.lib.perm.acl.cache import RoleCache
from api.lib.perm.acl.cache import User
from api.lib.perm.acl.cache import UserCache
from api.lib.perm.acl.resp_format import ErrFormat
from api.lib.perm.auth import auth_abandoned
from api.lib.perm.auth import auth_with_app_token
from api.models.acl import Role
from api.resource import APIView
class LoginView(APIView):
url_prefix = "/login"
@args_required("username")
@args_required("password")
@auth_abandoned
@args_validate(User)
def post(self):
username = request.values.get("username") or request.values.get("email")
password = request.values.get("password")
_role = None
if current_app.config.get('AUTH_WITH_LDAP'):
user, authenticated = User.query.authenticate_with_ldap(username, password)
else:
user, authenticated = User.query.authenticate(username, password)
if not user:
_role, authenticated = Role.query.authenticate(username, password)
if not user and not _role:
return abort(401, ErrFormat.user_not_found.format(username))
if not authenticated:
return abort(401, ErrFormat.invalid_password)
if user:
login_user(user)
user.update(has_logined=True, last_login=datetime.datetime.now())
token = jwt.encode({
'sub': user.email,
'iat': datetime.datetime.now(),
'exp': datetime.datetime.now() + datetime.timedelta(minutes=24 * 60 * 7)},
current_app.config['SECRET_KEY'])
username = username.split("@")[0]
user_info = ACLManager.get_user_info(username)
session["acl"] = dict(uid=user_info.get("uid"),
avatar=user.avatar if user else user_info.get("avatar"),
userId=user_info.get("uid"),
rid=user_info.get("rid"),
userName=user_info.get("username"),
nickName=user_info.get("nickname"),
parentRoles=user_info.get("parents"),
childRoles=user_info.get("children"),
roleName=user_info.get("role"))
session["uid"] = user_info.get("uid")
return self.jsonify(token=token.decode() if six.PY2 else token, username=username)
else:
return self.jsonify(username=username)
class AuthWithKeyView(APIView):
url_prefix = "/auth_with_key"
@args_required("key")
@args_required("secret")
@args_required("path")
@auth_abandoned
def post(self):
key = request.values.get('key')
secret = request.values.get('secret')
path = six.moves.urllib.parse.urlparse(request.values.get('path')).path
payload = request.values.get('payload') or {}
payload.pop('_key', None)
payload.pop('_secret', None)
req_args = [str(payload[k]) for k in sorted(payload.keys())]
user, authenticated = User.query.authenticate_with_key(key, secret, req_args, path)
if user:
role = RoleCache.get_by_name(None, user.username)
role or abort(404, ErrFormat.role_not_found.format(user.username))
user = user.to_dict()
else:
role, authenticated = Role.query.authenticate_with_key(key, secret, req_args, path)
user = role and role.to_dict() or {}
can_proxy = True if role and role.is_app_admin else False
if can_proxy and request.values.get('proxy'):
role = RoleCache.get_by_name(None, request.values.get('proxy'))
role or abort(404, ErrFormat.role_not_found.format(request.values.get('proxy')))
user = role and role.to_dict() or {}
user['rid'] = role and role.id
user.pop('password', None)
user.pop('key', None)
user.pop('secret', None)
if not user.get('username'):
user['username'] = user.get('name')
return self.jsonify(user=user,
authenticated=authenticated,
rid=role and role.id,
can_proxy=can_proxy)
class AuthWithTokenView(APIView):
url_prefix = ("/auth_with_token", "/req_token")
@auth_with_app_token
def get(self):
username = request.values.get('username')
token = jwt.encode({
'sub': username,
'iat': datetime.datetime.now(),
'exp': datetime.datetime.now() + datetime.timedelta(minutes=2 * 60)},
current_app.config['SECRET_KEY'])
return self.jsonify(token=token)
@args_required("token")
@auth_with_app_token
def post(self):
token = request.values.get('token')
user = None
try:
data = jwt.decode(token, current_app.config['SECRET_KEY'], algorithms=['HS256'])
authenticated = True
user = UserCache.get(data.get('sub'))
if not user:
authenticated = False
except jwt.ExpiredSignatureError:
authenticated = False
except (jwt.InvalidTokenError, Exception) as e:
current_app.logger.error(str(e))
authenticated = False
if user is not None:
role = RoleCache.get_by_name(None, user.username)
role or abort(404, ErrFormat.role_not_found.format(user.username))
user = user.to_dict()
user['rid'] = role and role.id
user.pop('password', None)
user.pop('key', None)
user.pop('secret', None)
return self.jsonify(user=user,
authenticated=authenticated)
class LogoutView(APIView):
url_prefix = "/logout"
@auth_abandoned
def post(self):
logout_user()
self.jsonify(code=200)

137
cmdb-api/api/views/acl/permission.py
View File

@ -1,10 +1,17 @@
# -*- coding:utf-8 -*-
from flask import abort
from flask import current_app
from flask import request
from api.lib.decorator import args_required
from api.lib.perm.acl.acl import ACLManager
from api.lib.perm.acl.cache import AppCache
from api.lib.perm.acl.permission import PermissionCRUD
from api.lib.perm.acl.resp_format import ErrFormat
from api.lib.perm.auth import auth_only_for_acl
from api.lib.perm.auth import auth_with_app_token
from api.lib.utils import handle_arg_list
from api.resource import APIView
@ -12,29 +19,149 @@ from api.resource import APIView
class ResourcePermissionView(APIView):
url_prefix = ("/resources/<int:resource_id>/permissions", "/resource_groups/<int:group_id>/permissions")
@auth_with_app_token
def get(self, resource_id=None, group_id=None):
return self.jsonify(PermissionCRUD.get_all(resource_id, group_id))
need_users = request.values.get('need_users', 1) in current_app.config.get('BOOL_TRUE')
return self.jsonify(PermissionCRUD.get_all(resource_id, group_id, need_users=need_users))
class ResourcePermission2View(APIView):
url_prefix = "/resource/permissions"
@args_required('resource_name')
@args_required('resource_type_name')
@auth_with_app_token
def get(self):
resource_name = request.values.get('resource_name')
resource_type_name = request.values.get('resource_type_name')
app_id = request.values.get('app_id')
return self.jsonify(PermissionCRUD.get_all2(resource_name, resource_type_name, app_id))
class RolePermissionGrantView(APIView):
url_prefix = ('/roles/<int:rid>/resources/<int:resource_id>/grant',
'/roles/<int:rid>/resource_groups/<int:group_id>/grant')
'/roles/<int:rid>/resource_groups/<int:group_id>/grant',
'/roles/<int:rid>/resources/batch/grant2', # by names
)
@args_required('perms')
@auth_only_for_acl
def post(self, rid, resource_id=None, group_id=None):
perms = handle_arg_list(request.values.get("perms"))
if "batch" in request.url:
resource_ids = request.values.get('resource_ids')
perm_map = request.values.get('perm_map')
resource_names = request.values.get('resource_names')
resource_type_id = request.values.get('resource_type_id')
app = AppCache.get(request.values.get('app_id'))
PermissionCRUD.batch_grant_by_resource_names(rid, perms, resource_type_id, resource_names,
resource_ids, perm_map, app_id=app and app.id)
return self.jsonify(rid=rid, resource_names=resource_names, resource_type_id=resource_type_id, perms=perms)
PermissionCRUD.grant(rid, perms, resource_id=resource_id, group_id=group_id)
return self.jsonify(rid=rid, resource_id=resource_id, group_id=group_id, perms=perms)
class RolePermissionGrant2View(APIView):
url_prefix = ('/roles/<int:rid>/resources/<int:resource_id>/grant2',)
def post(self, rid, resource_id):
if not ACLManager(request.values.get('app_id')).has_permission(None, None, 'grant', resource_id):
return abort(403, ErrFormat.no_permission2)
perms = handle_arg_list(request.values.get("perms"))
PermissionCRUD.grant(rid, perms, resource_id=resource_id)
return self.jsonify(rid=rid, resource_id=resource_id, perms=perms)
class RolePermissionBatchGrantView(APIView):
url_prefix = ('/roles/<int:rid>/resources/batch/grant',
'/roles/<int:rid>/resource_groups/batch/grant')
@auth_only_for_acl
def post(self, rid):
resource_ids = request.values.get('resource_ids')
group_ids = request.values.get('group_ids')
perms = handle_arg_list(request.values.get("perms"))
if resource_ids and isinstance(resource_ids, list):
for resource_id in resource_ids[:-1]:
PermissionCRUD.grant(rid, perms, resource_id=resource_id, group_id=None, rebuild=False)
PermissionCRUD.grant(rid, perms, resource_id=resource_ids[-1], group_id=None, rebuild=True)
if group_ids and isinstance(group_ids, list):
for group_id in group_ids[:-1]:
PermissionCRUD.grant(rid, perms, resource_id=None, group_id=group_id, rebuild=False)
PermissionCRUD.grant(rid, perms, resource_id=None, group_id=group_ids[-1], rebuild=True)
return self.jsonify(rid=rid, resource_ids=resource_ids, group_ids=group_ids, perms=perms)
class RolePermissionRevokeView(APIView):
url_prefix = ('/roles/<int:rid>/resources/<int:resource_id>/revoke',
'/roles/<int:rid>/resource_groups/<int:group_id>/revoke')
'/roles/<int:rid>/resource_groups/<int:group_id>/revoke',
'/roles/<int:rid>/resources/batch/revoke2', # by names
)
@args_required('perms')
@auth_only_for_acl
def post(self, rid, resource_id=None, group_id=None):
perms = handle_arg_list(request.values.get("perms"))
if "batch" in request.url:
resource_names = request.values.get('resource_names')
resource_type_id = request.values.get('resource_type_id')
resource_ids = request.values.get('resource_ids')
perm_map = request.values.get('perm_map')
app = AppCache.get(request.values.get('app_id'))
PermissionCRUD.batch_revoke_by_resource_names(rid, perms, resource_type_id, resource_names,
resource_ids, perm_map, app_id=app and app.id)
return self.jsonify(rid=rid, resource_names=resource_names, resource_type_id=resource_type_id, perms=perms)
PermissionCRUD.revoke(rid, perms, resource_id=resource_id, group_id=group_id)
return self.jsonify(rid=rid, resource_id=resource_id, group_id=group_id, perms=perms)
class RolePermissionRevoke2View(APIView):
url_prefix = ('/roles/<int:rid>/resources/<int:resource_id>/revoke2',
'/roles/<int:rid>/resource_groups/<int:group_id>/revoke2',)
def post(self, rid, resource_id=None, group_id=None):
if not ACLManager(request.values.get('app_id')).has_permission(None, None, 'grant', resource_id):
return abort(403, ErrFormat.no_permission2)
perms = handle_arg_list(request.values.get("perms"))
PermissionCRUD.revoke(rid, perms, resource_id=resource_id, group_id=group_id)
return self.jsonify(rid=rid, resource_id=resource_id, perms=perms)
class RolePermissionBatchRevokeView(APIView):
url_prefix = ('/roles/<int:rid>/resources/batch/revoke',
'/roles/<int:rid>/resource_groups/batch/revoke')
@auth_only_for_acl
def post(self, rid):
resource_ids = request.values.get('resource_ids')
group_ids = request.values.get('group_ids')
perms = handle_arg_list(request.values.get("perms"))
if resource_ids and isinstance(resource_ids, list):
for resource_id in resource_ids[:-1]:
PermissionCRUD.revoke(rid, perms, resource_id=resource_id, group_id=None, rebuild=False)
PermissionCRUD.revoke(rid, perms, resource_id=resource_ids[-1], group_id=None, rebuild=True)
if group_ids and isinstance(group_ids, list):
for group_id in group_ids[:-1]:
PermissionCRUD.revoke(rid, perms, resource_id=None, group_id=group_id, rebuild=False)
PermissionCRUD.revoke(rid, perms, resource_id=None, group_id=group_ids[-1], rebuild=True)
return self.jsonify(rid=rid, resource_ids=resource_ids, group_ids=group_ids, perms=perms)

43
cmdb-api/api/views/acl/resources.py
View File

@ -1,12 +1,16 @@
# -*- coding:utf-8 -*-
from flask import g
from flask import request
from api.lib.decorator import args_required
from api.lib.decorator import args_validate
from api.lib.perm.acl import validate_app
from api.lib.perm.acl.resource import ResourceCRUD
from api.lib.perm.acl.resource import ResourceGroupCRUD
from api.lib.perm.acl.resource import ResourceTypeCRUD
from api.lib.perm.auth import auth_only_for_acl
from api.lib.perm.auth import auth_with_app_token
from api.lib.utils import get_page
from api.lib.utils import get_page_size
from api.lib.utils import handle_arg_list
@ -16,8 +20,8 @@ from api.resource import APIView
class ResourceTypeView(APIView):
url_prefix = ("/resource_types", "/resource_types/<int:type_id>")
@args_required('app_id')
@validate_app
@auth_with_app_token
def get(self):
page = get_page(request.values.get("page", 1))
page_size = get_page_size(request.values.get("page_size"))
@ -33,9 +37,10 @@ class ResourceTypeView(APIView):
id2perms=id2perms)
@args_required('name')
@args_required('app_id')
@args_required('perms')
@validate_app
@auth_only_for_acl
@args_validate(ResourceTypeCRUD.cls, exclude_args=['app_id'])
def post(self):
name = request.values.get('name')
app_id = request.values.get('app_id')
@ -46,11 +51,14 @@ class ResourceTypeView(APIView):
return self.jsonify(rt.to_dict())
@auth_only_for_acl
@args_validate(ResourceTypeCRUD.cls, exclude_args=['app_id'])
def put(self, type_id):
rt = ResourceTypeCRUD.update(type_id, **request.values)
return self.jsonify(rt.to_dict())
@auth_only_for_acl
def delete(self, type_id):
ResourceTypeCRUD.delete(type_id)
@ -60,6 +68,7 @@ class ResourceTypeView(APIView):
class ResourceTypePermsView(APIView):
url_prefix = "/resource_types/<int:type_id>/perms"
@auth_with_app_token
def get(self, type_id):
return self.jsonify(ResourceTypeCRUD.get_perms(type_id))
@ -67,36 +76,43 @@ class ResourceTypePermsView(APIView):
class ResourceView(APIView):
url_prefix = ("/resources", "/resources/<int:resource_id>")
@args_required('app_id')
@validate_app
@auth_with_app_token
def get(self):
page = get_page(request.values.get("page", 1))
page_size = get_page_size(request.values.get("page_size"))
q = request.values.get('q')
u = request.values.get('u')
resource_type_id = request.values.get('resource_type_id')
app_id = request.values.get('app_id')
numfound, res = ResourceCRUD.search(q, app_id, resource_type_id, page, page_size)
numfound, res = ResourceCRUD.search(q, u, app_id, resource_type_id, page, page_size)
return self.jsonify(numfound=numfound,
page=page,
page_size=page_size,
resources=[i.to_dict() for i in res])
resources=res)
@args_required('name')
@args_required('type_id')
@args_required('app_id')
@validate_app
@auth_only_for_acl
@args_validate(ResourceCRUD.cls, exclude_args=['app_id'])
def post(self):
name = request.values.get('name')
type_id = request.values.get('type_id')
app_id = request.values.get('app_id')
uid = request.values.get('uid')
if not uid and hasattr(g, "user") and hasattr(g.user, "uid"):
uid = g.user.uid
resource = ResourceCRUD.add(name, type_id, app_id)
resource = ResourceCRUD.add(name, type_id, app_id, uid)
return self.jsonify(resource.to_dict())
@args_required('name')
@auth_only_for_acl
@args_validate(ResourceCRUD.cls, exclude_args=['app_id'])
def put(self, resource_id):
name = request.values.get('name')
@ -104,6 +120,7 @@ class ResourceView(APIView):
return self.jsonify(resource.to_dict())
@auth_only_for_acl
def delete(self, resource_id):
ResourceCRUD.delete(resource_id)
@ -113,15 +130,16 @@ class ResourceView(APIView):
class ResourceGroupView(APIView):
url_prefix = ("/resource_groups", "/resource_groups/<int:group_id>")
@args_required('app_id')
@validate_app
@auth_with_app_token
def get(self):
page = get_page(request.values.get("page", 1))
page_size = get_page_size(request.values.get("page_size"))
q = request.values.get('q')
app_id = request.values.get('app_id')
resource_type_id = request.values.get('resource_type_id')
numfound, res = ResourceGroupCRUD.search(q, app_id, page, page_size)
numfound, res = ResourceGroupCRUD.search(q, app_id, resource_type_id, page, page_size)
return self.jsonify(numfound=numfound,
page=page,
@ -130,8 +148,9 @@ class ResourceGroupView(APIView):
@args_required('name')
@args_required('type_id')
@args_required('app_id')
@validate_app
@auth_only_for_acl
@args_validate(ResourceGroupCRUD.cls, exclude_args=['app_id'])
def post(self):
name = request.values.get('name')
type_id = request.values.get('type_id')
@ -142,6 +161,8 @@ class ResourceGroupView(APIView):
return self.jsonify(group.to_dict())
@args_required('items')
@auth_only_for_acl
@args_validate(ResourceGroupCRUD.cls, exclude_args=['app_id'])
def put(self, group_id):
items = handle_arg_list(request.values.get("items"))
@ -151,6 +172,7 @@ class ResourceGroupView(APIView):
return self.jsonify(items)
@auth_only_for_acl
def delete(self, group_id):
ResourceGroupCRUD.delete(group_id)
@ -160,6 +182,7 @@ class ResourceGroupView(APIView):
class ResourceGroupItemsView(APIView):
url_prefix = "/resource_groups/<int:group_id>/items"
@auth_with_app_token
def get(self, group_id):
items = ResourceGroupCRUD.get_items(group_id)

136
cmdb-api/api/views/acl/role.py
View File

@ -1,12 +1,21 @@
# -*- coding:utf-8 -*-
from flask import abort
from flask import current_app
from flask import g
from flask import request
from api.lib.decorator import args_required
from api.lib.decorator import args_validate
from api.lib.perm.acl import validate_app
from api.lib.perm.acl.acl import is_app_admin
from api.lib.perm.acl.cache import AppCache
from api.lib.perm.acl.cache import RoleCache
from api.lib.perm.acl.resp_format import ErrFormat
from api.lib.perm.acl.role import RoleCRUD
from api.lib.perm.acl.role import RoleRelationCRUD
from api.lib.perm.auth import auth_only_for_acl
from api.lib.perm.auth import auth_with_app_token
from api.lib.utils import get_page
from api.lib.utils import get_page_size
from api.resource import APIView
@ -15,43 +24,56 @@ from api.resource import APIView
class RoleView(APIView):
url_prefix = ("/roles", "/roles/<int:rid>")
@args_required('app_id')
@validate_app
@auth_with_app_token
def get(self):
page = get_page(request.values.get("page", 1))
page_size = get_page_size(request.values.get("page_size"))
q = request.values.get('q')
app_id = request.values.get('app_id')
is_all = request.values.get('is_all', True)
is_all = True if is_all in current_app.config.get("BOOL_TRUE") else False
user_role = request.values.get('user_role', True)
user_only = request.values.get('user_only', False)
user_role = True if user_role in current_app.config.get("BOOL_TRUE") else False
user_only = True if user_only in current_app.config.get("BOOL_TRUE") else False
numfound, roles = RoleCRUD.search(q, app_id, page, page_size, user_role)
numfound, roles = RoleCRUD.search(q, app_id, page, page_size, user_role, is_all, user_only)
id2parents = RoleRelationCRUD.get_parents([i.id for i in roles])
id2parents = RoleRelationCRUD.get_parents([i.id for i in roles], app_id=app_id)
roles = [i.to_dict() for i in roles]
for i in roles:
i.pop('password', None)
return self.jsonify(numfound=numfound,
page=page,
page_size=page_size,
id2parents=id2parents,
roles=[i.to_dict() for i in roles])
roles=roles)
@args_required('name')
@args_required('app_id')
@validate_app
@auth_with_app_token
@args_validate(RoleCRUD.cls, exclude_args=['app_id'])
def post(self):
name = request.values.get('name')
app_id = request.values.get('app_id')
is_app_admin = request.values.get('is_app_admin', False)
password = request.values.get('password')
_is_app_admin = request.values.get('is_app_admin', False)
role = RoleCRUD.add_role(name, app_id, is_app_admin=is_app_admin)
role = RoleCRUD.add_role(name, app_id, password=password, is_app_admin=_is_app_admin)
return self.jsonify(role.to_dict())
@auth_only_for_acl
@args_validate(RoleCRUD.cls, exclude_args=['app_id'])
def put(self, rid):
role = RoleCRUD.update_role(rid, **request.values)
return self.jsonify(role.to_dict())
@auth_only_for_acl
def delete(self, rid):
RoleCRUD.delete_role(rid)
@ -59,19 +81,97 @@ class RoleView(APIView):
class RoleRelationView(APIView):
url_prefix = "/roles/<int:child_id>/parents"
url_prefix = ("/roles/<int:rid>/parents", "/roles/<int:rid>/users", "/roles/<int:rid>/children")
@auth_with_app_token
@validate_app
def get(self, rid):
app_id = request.values.get('app_id')
app = AppCache.get(app_id)
if app and app.name == "acl":
app_id = None # global
users = RoleRelationCRUD.get_users_by_rid(rid, app_id)
return self.jsonify(users=users)
@auth_only_for_acl
@validate_app
@args_validate(RoleRelationCRUD.cls, exclude_args=['app_id'])
def post(self, rid):
app_id = request.values.get('app_id')
app = AppCache.get(app_id)
if app and app.name == "acl":
app_id = None # global
role = RoleCache.get(rid) or abort(400, ErrFormat.role_not_found.format("id={}".format(rid)))
if request.values.get('parent_id'):
parent_id = request.values.get('parent_id')
res = RoleRelationCRUD.add(role, parent_id, [rid], app_id)
return self.jsonify(res)
elif request.values.get("child_ids") and isinstance(request.values['child_ids'], list):
res = RoleRelationCRUD.add(role, rid, request.values['child_ids'], app_id)
return self.jsonify(res)
else:
return abort(400, ErrFormat.invalid_request)
@args_required('parent_id')
def post(self, child_id):
parent_id = request.values.get('parent_id')
res = RoleRelationCRUD.add(parent_id, child_id)
return self.jsonify(res.to_dict())
@args_required('parent_id')
def delete(self, child_id):
@auth_only_for_acl
@validate_app
def delete(self, rid):
parent_id = request.values.get('parent_id')
RoleRelationCRUD.delete2(parent_id, child_id)
app_id = request.values.get('app_id')
app = AppCache.get(app_id)
if app and app.name == "acl":
app_id = None # global
return self.jsonify(parent_id=parent_id, child_id=child_id)
RoleRelationCRUD.delete2(parent_id, rid, app_id)
return self.jsonify(parent_id=parent_id, child_id=rid)
class RoleResourcesView(APIView):
url_prefix = "/roles/<int:rid>/resources"
@auth_with_app_token
@validate_app
def get(self, rid):
resource_type_id = request.values.get('resource_type_id')
group_flat = request.values.get('group_flat', True)
res = RoleCRUD.recursive_resources(rid, request.values['app_id'], resource_type_id, group_flat, to_record=True)
return self.jsonify(res)
class RoleHasPermissionView(APIView):
url_prefix = "/roles/has_perm"
@args_required('resource_name')
@args_required('resource_type_name')
@args_required('perm')
@validate_app
@auth_with_app_token
def get(self):
if not request.values.get('rid'):
role = RoleCache.get_by_name(None, g.user.username)
role or abort(404, ErrFormat.role_not_found.format(g.user.username))
else:
role = RoleCache.get(int(request.values.get('rid')))
app_id = request.values.get('app_id')
if is_app_admin(app_id):
return self.jsonify(result=True)
resource_name = request.values.get('resource_name')
resource_type_name = request.values.get('resource_type_name')
perm = request.values.get('perm')
result = RoleCRUD.has_permission(role.id, resource_name, resource_type_name, app_id, perm)
return self.jsonify(result=result)

92
cmdb-api/api/views/acl/trigger.py Normal file
View File

@ -0,0 +1,92 @@
# -*- coding:utf-8 -*-
from flask import request
from api.lib.decorator import args_required
from api.lib.decorator import args_validate
from api.lib.perm.acl import validate_app
from api.lib.perm.acl.trigger import TriggerCRUD
from api.lib.perm.auth import auth_only_for_acl
from api.lib.perm.auth import auth_with_app_token
from api.resource import APIView
class TriggerView(APIView):
url_prefix = ("/triggers", "/triggers/<int:_id>")
@validate_app
@auth_with_app_token
def get(self):
return self.jsonify(TriggerCRUD.get(request.values.get('app_id')))
@args_required('name')
@args_required('resource_type_id')
@args_required('roles')
@args_required('permissions')
@validate_app
@auth_only_for_acl
@args_validate(TriggerCRUD.cls, exclude_args=['app_id'])
def post(self):
request.values.pop('_key', None)
request.values.pop('_secret', None)
trigger = TriggerCRUD.add(request.values.pop('app_id', None), **request.values)
return self.jsonify(trigger.to_dict())
@args_required('resource_type_id')
@args_required('roles')
@args_required('permissions')
@validate_app
@auth_only_for_acl
@args_validate(TriggerCRUD.cls, exclude_args=['app_id'])
def put(self, _id):
request.values.pop('_key', None)
request.values.pop('_secret', None)
trigger = TriggerCRUD.update(_id, **request.values)
return self.jsonify(trigger.to_dict())
@auth_only_for_acl
def delete(self, _id):
TriggerCRUD.delete(_id)
return self.jsonify(id=_id)
class TriggerResourceView(APIView):
url_prefix = "/triggers/resources"
@validate_app
@auth_with_app_token
@args_required("resource_type_id")
def post(self):
app_id = request.values.get('app_id')
resource_type_id = request.values.get('resource_type_id')
wildcard = request.values.get('pattern')
uid = request.values.get('owner')
resources = TriggerCRUD.get_resources(app_id, resource_type_id, wildcard, uid)
resources = [i.to_dict() for i in resources]
return self.jsonify(resources)
class TriggerApplyView(APIView):
url_prefix = "/triggers/<int:_id>/apply"
@auth_only_for_acl
def post(self, _id):
TriggerCRUD.apply(_id)
return self.jsonify(id=_id)
class TriggerCancelView(APIView):
url_prefix = "/triggers/<int:_id>/cancel"
@auth_only_for_acl
def post(self, _id):
TriggerCRUD.cancel(_id)
return self.jsonify(id=_id)

113
cmdb-api/api/views/acl/user.py
View File

@ -1,13 +1,25 @@
# -*- coding:utf-8 -*-
import requests
from flask import abort
from flask import current_app
from flask import g
from flask import request
from flask import session
from flask_login import current_user
from api.lib.decorator import args_required
from api.lib.decorator import args_validate
from api.lib.perm.acl.acl import ACLManager
from api.lib.perm.acl.acl import role_required
from api.lib.perm.acl.audit import AuditCRUD, AuditOperateType
from api.lib.perm.acl.cache import AppCache
from api.lib.perm.acl.cache import UserCache
from api.lib.perm.acl.resp_format import ErrFormat
from api.lib.perm.acl.role import RoleRelationCRUD
from api.lib.perm.acl.user import UserCRUD
from api.lib.perm.auth import auth_with_app_token
from api.lib.utils import get_page
from api.lib.utils import get_page_size
from api.resource import APIView
@ -16,25 +28,57 @@ from api.resource import APIView
class GetUserInfoView(APIView):
url_prefix = "/users/info"
@auth_with_app_token
def get(self):
name = session.get("CAS_USERNAME") or current_user.nickname
role = dict(permissions=session.get("acl", {}).get("parentRoles", []))
avatar = current_user.avatar
app_id = request.values.get('app_id')
if not app_id:
name = session.get("acl", {}).get("userName") or session.get("CAS_USERNAME") or \
current_user.username or request.values.get('username')
else:
return self.jsonify(result=dict(name=name,
role=role,
avatar=avatar))
name = request.values.get('username')
current_app.logger.info("get user info for1: app_id: {0}, name: {1}".format(request.values.get('app_id'), name))
user_info = ACLManager().get_user_info(name, request.values.get('app_id'))
current_app.logger.info("get user info for2: {}".format(user_info))
result = dict(name=user_info.get('nickname') or name,
username=user_info.get('username') or name,
email=user_info.get('email'),
uid=user_info.get('uid'),
rid=user_info.get('rid'),
role=dict(permissions=user_info.get('parents')),
avatar=user_info.get('avatar'))
current_app.logger.info("get user info for3: {}".format(result))
return self.jsonify(result=result)
class GetUserKeySecretView(APIView):
url_prefix = "/users/secret"
@auth_with_app_token
def get(self):
if not request.values.get('app_id'):
name = session.get("acl", {}).get("userName") or session.get("CAS_USERNAME") or current_user.username
else:
name = request.values.get('username')
user = UserCache.get(name) or abort(404, ErrFormat.user_not_found.format(name))
return self.jsonify(key=user.key, secret=user.secret)
class UserView(APIView):
url_prefix = ("/users", "/users/<int:uid>")
@auth_with_app_token
def get(self):
page = get_page(request.values.get('page', 1))
page_size = get_page_size(request.values.get('page_size'))
q = request.values.get("q")
numfound, users = UserCRUD.search(q, page, page_size)
id2parents = RoleRelationCRUD.get_parents(uids=[i.uid for i in users])
id2parents = RoleRelationCRUD.get_parents(uids=[i.uid for i in users], all_app=True)
users = [i.to_dict() for i in users]
for u in users:
@ -50,22 +94,49 @@ class UserView(APIView):
@args_required('username')
@args_required('email')
@role_required("acl_admin")
@args_validate(UserCRUD.cls)
def post(self):
request.values.pop('_key', None)
request.values.pop('_secret', None)
user = UserCRUD.add(**request.values)
return self.jsonify(user.to_dict())
@role_required("acl_admin")
@args_validate(UserCRUD.cls)
def put(self, uid):
request.values.pop('_key', None)
request.values.pop('_secret', None)
user = UserCRUD.update(uid, **request.values)
return self.jsonify(user.to_dict())
@role_required("acl_admin")
def delete(self, uid):
if g.user.uid == uid:
return abort(400, ErrFormat.invalid_operation)
UserCRUD.delete(uid)
return self.jsonify(uid=uid)
class UserOnTheJobView(APIView):
url_prefix = ("/users/employee",)
@auth_with_app_token
def get(self):
if current_app.config.get('HR_URI'):
try:
return self.jsonify(requests.get(current_app.config["HR_URI"]).json())
except:
return abort(400, ErrFormat.invalid_request)
else:
return self.jsonify(UserCRUD.get_employees())
class UserResetKeySecretView(APIView):
url_prefix = "/users/reset_key_secret"
@ -76,3 +147,31 @@ class UserResetKeySecretView(APIView):
def put(self):
return self.post()
class UserResetPasswordView(APIView):
url_prefix = "/users/reset_password"
@auth_with_app_token
@args_required('username')
@args_required('password')
@args_validate(UserCRUD.cls, exclude_args=['app_id'])
def post(self):
if request.values.get('app_id'):
app = AppCache.get(request.values['app_id'])
if app.name not in ('cas-server', 'acl'):
return abort(403, ErrFormat.invalid_request)
elif hasattr(g, 'user'):
if g.user.username != request.values['username']:
return abort(403, ErrFormat.invalid_request)
else:
return abort(400, ErrFormat.invalid_operation)
user = UserCache.get(request.values['username'])
user or abort(404, ErrFormat.user_not_found.format(request.values['username']))
UserCRUD.update(user.uid, password=request.values['password'])
return self.jsonify(code=200)

21
cmdb-api/api/views/cmdb/attribute.py
View File

@ -6,9 +6,9 @@ from flask import current_app
from flask import request
from api.lib.cmdb.attribute import AttributeManager
from api.lib.cmdb.const import RoleEnum
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.decorator import args_required
from api.lib.perm.acl.acl import role_required
from api.lib.decorator import args_validate
from api.lib.utils import get_page
from api.lib.utils import get_page_size
from api.lib.utils import handle_arg_list
@ -42,24 +42,30 @@ class AttributeView(APIView):
attr_dict = attr_manager.get_attribute_by_name(attr_name)
if attr_dict is None:
attr_dict = attr_manager.get_attribute_by_alias(attr_name)
if not attr_dict:
return abort(404, ErrFormat.attribute_not_found.format("name={}".format(attr_name)))
elif attr_id is not None:
attr_dict = attr_manager.get_attribute_by_id(attr_id)
if attr_dict is not None:
return self.jsonify(attribute=attr_dict)
abort(404, "Attribute is not found")
@role_required(RoleEnum.CONFIG)
if not attr_dict:
return abort(404, ErrFormat.attribute_not_found.format("name={}".format(attr_name)))
return self.jsonify(attribute=attr_dict)
@args_required("name")
@args_validate(AttributeManager.cls)
def post(self):
choice_value = handle_arg_list(request.values.get("choice_value"))
params = request.values
params["choice_value"] = choice_value
current_app.logger.debug(params)
attr_id = AttributeManager.add(**params)
return self.jsonify(attr_id=attr_id)
@role_required(RoleEnum.CONFIG)
@args_validate(AttributeManager.cls)
def put(self, attr_id):
choice_value = handle_arg_list(request.values.get("choice_value"))
params = request.values
@ -68,7 +74,6 @@ class AttributeView(APIView):
AttributeManager().update(attr_id, **params)
return self.jsonify(attr_id=attr_id)
@role_required(RoleEnum.CONFIG)
def delete(self, attr_id):
attr_name = AttributeManager.delete(attr_id)
return self.jsonify(message="attribute {0} deleted".format(attr_name))

236
cmdb-api/api/views/cmdb/auto_discovery.py Normal file
View File

@ -0,0 +1,236 @@
# -*- coding:utf-8 -*-
import json
from io import BytesIO
from flask import abort
from flask import current_app
from flask import g
from flask import request
from api.lib.cmdb.auto_discovery.auto_discovery import AutoDiscoveryCICRUD
from api.lib.cmdb.auto_discovery.auto_discovery import AutoDiscoveryCITypeCRUD
from api.lib.cmdb.auto_discovery.auto_discovery import AutoDiscoveryHTTPManager
from api.lib.cmdb.auto_discovery.auto_discovery import AutoDiscoveryRuleCRUD
from api.lib.cmdb.auto_discovery.auto_discovery import AutoDiscoverySNMPManager
from api.lib.cmdb.auto_discovery.const import DEFAULT_HTTP
from api.lib.cmdb.const import PermEnum
from api.lib.cmdb.const import ResourceTypeEnum
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.cmdb.search import SearchError
from api.lib.cmdb.search.ci import search
from api.lib.decorator import args_required
from api.lib.decorator import args_validate
from api.lib.perm.acl.acl import has_perm_from_args
from api.lib.utils import AESCrypto
from api.lib.utils import get_page
from api.lib.utils import get_page_size
from api.lib.utils import handle_arg_list
from api.resource import APIView
class AutoDiscoveryRuleView(APIView):
url_prefix = ("/adr", "/adr/<int:adr_id>")
def get(self):
_, res = AutoDiscoveryRuleCRUD.search(page=1, page_size=100000, **request.values)
rebuild = False
exists = {i['name'] for i in res}
for i in DEFAULT_HTTP:
if i['name'] not in exists:
AutoDiscoveryRuleCRUD().add(**i)
rebuild = True
if rebuild:
_, res = AutoDiscoveryRuleCRUD.search(page=1, page_size=100000, **request.values)
return self.jsonify(res)
@args_required("name", value_required=True)
@args_validate(AutoDiscoveryRuleCRUD.cls)
def post(self):
return self.jsonify(AutoDiscoveryRuleCRUD().add(**request.values).to_dict())
@args_validate(AutoDiscoveryRuleCRUD.cls)
def put(self, adr_id):
return self.jsonify(AutoDiscoveryRuleCRUD().update(adr_id, **request.values).to_dict())
def delete(self, adr_id):
AutoDiscoveryRuleCRUD().delete(adr_id)
return self.jsonify(adr_id=adr_id)
class AutoDiscoveryRuleTemplateFileView(APIView):
url_prefix = ("/adr/template/import/file", "/adr/template/export/file")
def get(self): # export
adr_tpt = AutoDiscoveryRuleCRUD().get_by_inner()
adr_tpt = dict(auto_discovery_rules=adr_tpt)
bf = BytesIO()
bf.write(bytes(json.dumps(adr_tpt).encode('utf-8')))
bf.seek(0)
return self.send_file(bf,
as_attachment=True,
attachment_filename="cmdb_auto_discovery.json",
mimetype='application/json',
cache_timeout=0)
def post(self):
f = request.files.get('file')
if f is None:
return abort(400, ErrFormat.argument_file_not_found)
content = f.read()
try:
content = json.loads(content)
except:
return abort(400, ErrFormat.invalid_json)
tpt = content.get('auto_discovery_rules')
AutoDiscoveryRuleCRUD().import_template(tpt)
return self.jsonify(code=200)
class AutoDiscoveryRuleHTTPView(APIView):
url_prefix = ("/adr/http/<string:name>/categories", "/adr/http/<string:name>/attributes",
"/adr/snmp/<string:name>/attributes")
def get(self, name):
if "snmp" in request.url:
return self.jsonify(AutoDiscoverySNMPManager.get_attributes())
if "attributes" in request.url:
category = request.values.get('category')
return self.jsonify(AutoDiscoveryHTTPManager.get_attributes(name, category))
return self.jsonify(AutoDiscoveryHTTPManager.get_categories(name))
class AutoDiscoveryCITypeView(APIView):
url_prefix = ("/adt/ci_types/<int:type_id>", "/adt/<int:adt_id>")
def get(self, type_id):
_, res = AutoDiscoveryCITypeCRUD.search(page=1, page_size=100000, type_id=type_id, **request.values)
for i in res:
if isinstance(i.get("extra_option"), dict) and i['extra_option'].get('secret'):
if not (g.user.username == "cmdb_agent" or g.user.uid == i['uid']):
i['extra_option'].pop('secret', None)
else:
i['extra_option']['secret'] = AESCrypto.decrypt(i['extra_option']['secret'])
return self.jsonify(res)
@args_validate(AutoDiscoveryCITypeCRUD.cls)
def post(self, type_id):
if not request.values.get('interval'):
request.values.pop('interval', None)
return self.jsonify(AutoDiscoveryCITypeCRUD().add(type_id=type_id, **request.values).to_dict())
@args_validate(AutoDiscoveryCITypeCRUD.cls)
def put(self, adt_id):
if not request.values.get('interval'):
request.values.pop('interval', None)
return self.jsonify(AutoDiscoveryCITypeCRUD().update(adt_id, **request.values).to_dict())
def delete(self, adt_id):
AutoDiscoveryCITypeCRUD().delete(adt_id)
return self.jsonify(adt_id=adt_id)
class AutoDiscoveryCIView(APIView):
url_prefix = ("/adc", "/adc/<int:adc_id>", "/adc/ci_types/<int:type_id>/attributes", "/adc/ci_types")
def get(self, type_id=None):
if "attributes" in request.url:
return self.jsonify(AutoDiscoveryCICRUD.get_attributes_by_type_id(type_id))
elif "ci_types" in request.url:
need_other = request.values.get("need_other")
return self.jsonify(AutoDiscoveryCICRUD.get_ci_types(need_other))
page = get_page(request.values.pop('page', 1))
page_size = get_page_size(request.values.pop('page_size', None))
fl = handle_arg_list(request.values.get('fl'))
numfound, res = AutoDiscoveryCICRUD.search(page=page, page_size=page_size, fl=fl, **request.values)
return self.jsonify(page=page,
page_size=page_size,
numfound=numfound,
total=len(res),
result=res)
@args_validate(AutoDiscoveryCICRUD.cls)
@args_required("type_id")
@args_required("adt_id")
@args_required("instance")
def post(self):
request.values.pop("_key", None)
request.values.pop("_secret", None)
return self.jsonify(AutoDiscoveryCICRUD().upsert(**request.values).to_dict())
def put(self):
return self.post()
@has_perm_from_args("adc_id", ResourceTypeEnum.CI, PermEnum.DELETE, AutoDiscoveryCICRUD.get_type_name)
def delete(self, adc_id):
AutoDiscoveryCICRUD().delete(adc_id)
return self.jsonify(adc_id=adc_id)
class AutoDiscoveryCIDelete2View(APIView):
url_prefix = ("/adc",)
def delete(self):
type_id = request.values.get('type_id')
unique_value = request.values.get('unique_value')
AutoDiscoveryCICRUD.delete2(type_id, unique_value)
return self.jsonify(type_id=type_id, unique_value=unique_value)
class AutoDiscoveryCIAcceptView(APIView):
url_prefix = ("/adc/<int:adc_id>/accept",)
@has_perm_from_args("adc_id", ResourceTypeEnum.CI, PermEnum.ADD, AutoDiscoveryCICRUD.get_type_name)
def put(self, adc_id):
AutoDiscoveryCICRUD.accept(None, adc_id=adc_id)
return self.jsonify(adc_id=adc_id)
class AutoDiscoveryRuleSyncView(APIView):
url_prefix = ("/adt/sync",)
def get(self):
if g.user.username not in ("cmdb_agent", "worker", "admin"):
return abort(403)
oneagent_name = request.values.get('oneagent_name')
oneagent_id = request.values.get('oneagent_id')
last_update_at = request.values.get('last_update_at')
query = "{},oneagent_id:{}".format(oneagent_name, oneagent_id)
current_app.logger.info(query)
s = search(query)
try:
response, _, _, _, _, _ = s.search()
except SearchError as e:
import traceback
current_app.logger.error(traceback.format_exc())
return abort(400, str(e))
ci_id = response and response[0]["_id"]
rules, last_update_at = AutoDiscoveryCITypeCRUD.get(ci_id, oneagent_id, last_update_at)
return self.jsonify(rules=rules, last_update_at=last_update_at)

12
cmdb-api/api/views/cmdb/c_stats.py Normal file
View File

@ -0,0 +1,12 @@
# -*- coding:utf-8 -*-
from api.lib.cmdb.cache import CMDBCounterCache
from api.resource import APIView
class CMDBStatisticsView(APIView):
url_prefix = "/statistics"
def get(self):
return self.jsonify(CMDBCounterCache.get())

73
cmdb-api/api/views/cmdb/ci.py
View File

@ -9,14 +9,14 @@ from flask import request
from api.lib.cmdb.cache import CITypeCache
from api.lib.cmdb.ci import CIManager
from api.lib.cmdb.ci import CIRelationManager
from api.lib.cmdb.const import ExistPolicy
from api.lib.cmdb.const import ResourceTypeEnum, PermEnum
from api.lib.cmdb.const import RetKey
from api.lib.cmdb.perms import has_perm_for_ci
from api.lib.cmdb.search import SearchError
from api.lib.cmdb.search.ci.db.search import Search as SearchFromDB
from api.lib.cmdb.search.ci.es.search import Search as SearchFromES
from api.lib.cmdb.search.ci import search
from api.lib.perm.acl.acl import has_perm_from_args
from api.lib.perm.auth import auth_abandoned
from api.lib.utils import get_page
from api.lib.utils import get_page_size
from api.lib.utils import handle_arg_list
@ -62,52 +62,63 @@ class CIView(APIView):
ret_key = RetKey.NAME
manager = CIManager()
ci = manager.get_ci_by_id_from_db(ci_id, ret_key=ret_key, fields=fields)
ci = manager.get_ci_by_id_from_db(ci_id, ret_key=ret_key, fields=fields, valid=True)
return self.jsonify(ci_id=ci_id, ci=ci)
@staticmethod
def _wrap_ci_dict():
ci_dict = dict()
for k, v in request.values.items():
if k != "ci_type" and not k.startswith("_"):
ci_dict[k] = v.strip() if isinstance(v, six.string_types) else v
ci_dict = {k: v.strip() if isinstance(v, six.string_types) else v for k, v in request.values.items()
if k != "ci_type" and not k.startswith("_")}
return ci_dict
@has_perm_from_args("ci_type", ResourceTypeEnum.CI, PermEnum.ADD, lambda x: CITypeCache.get(x).name)
@has_perm_for_ci("ci_type", ResourceTypeEnum.CI, PermEnum.ADD, lambda x: CITypeCache.get(x))
def post(self):
ci_type = request.values.get("ci_type")
_no_attribute_policy = request.values.get("_no_attribute_policy", ExistPolicy.IGNORE)
_no_attribute_policy = request.values.get("no_attribute_policy", ExistPolicy.IGNORE)
exist_policy = request.values.pop('exist_policy', None)
ci_dict = self._wrap_ci_dict()
manager = CIManager()
current_app.logger.debug(ci_dict)
ci_id = manager.add(ci_type,
exist_policy=ExistPolicy.REJECT,
_no_attribute_policy=_no_attribute_policy, **ci_dict)
exist_policy=exist_policy or ExistPolicy.REJECT,
_no_attribute_policy=_no_attribute_policy,
_is_admin=request.values.pop('__is_admin', False),
**ci_dict)
return self.jsonify(ci_id=ci_id)
@has_perm_from_args("ci_id", ResourceTypeEnum.CI, PermEnum.UPDATE, CIManager.get_type_name)
@has_perm_for_ci("ci_id", ResourceTypeEnum.CI, PermEnum.UPDATE, CIManager.get_type)
def put(self, ci_id=None):
args = request.values
current_app.logger.info(args)
ci_type = args.get("ci_type")
_no_attribute_policy = args.get("_no_attribute_policy", ExistPolicy.IGNORE)
_no_attribute_policy = args.get("no_attribute_policy", ExistPolicy.IGNORE)
ci_dict = self._wrap_ci_dict()
manager = CIManager()
if ci_id is not None:
manager.update(ci_id, **ci_dict)
manager.update(ci_id,
_is_admin=request.values.pop('__is_admin', False),
**ci_dict)
else:
ci_id = manager.add(ci_type,
exist_policy=ExistPolicy.REPLACE,
_no_attribute_policy=_no_attribute_policy,
_is_admin=request.values.pop('__is_admin', False),
**ci_dict)
return self.jsonify(ci_id=ci_id)
@has_perm_from_args("ci_id", ResourceTypeEnum.CI, PermEnum.DELETE, CIManager.get_type_name)
@has_perm_for_ci("ci_id", ResourceTypeEnum.CI, PermEnum.DELETE, CIManager.get_type)
def delete(self, ci_id):
manager = CIManager()
manager.delete(ci_id)
return self.jsonify(message="ok")
@ -116,13 +127,13 @@ class CIDetailView(APIView):
def get(self, ci_id):
_ci = CI.get_by_id(ci_id).to_dict()
return self.jsonify(**_ci)
class CISearchView(APIView):
url_prefix = ("/ci/s", "/ci/search")
@auth_abandoned
def get(self):
"""@params: q: query statement
fl: filter by column
@ -130,12 +141,12 @@ class CISearchView(APIView):
ret_key: id, name, alias
facet: statistic
"""
page = get_page(request.values.get("page", 1))
count = get_page_size(request.values.get("count") or request.values.get("page_size"))
query = request.values.get('q', "")
fl = handle_arg_list(request.values.get('fl', ""))
excludes = handle_arg_list(request.values.get('excludes', ""))
ret_key = request.values.get('ret_key', RetKey.NAME)
if ret_key not in (RetKey.NAME, RetKey.ALIAS, RetKey.ID):
ret_key = RetKey.NAME
@ -143,15 +154,18 @@ class CISearchView(APIView):
sort = request.values.get("sort")
start = time.time()
if current_app.config.get("USE_ES"):
s = SearchFromES(query, fl, facet, page, ret_key, count, sort)
else:
s = SearchFromDB(query, fl, facet, page, ret_key, count, sort)
s = search(query, fl, facet, page, ret_key, count, sort, excludes)
try:
response, counter, total, page, numfound, facet = s.search()
except SearchError as e:
return abort(400, str(e))
current_app.logger.debug("search time is :{0}".format(time.time() - start))
if request.values.get('need_children') in current_app.config.get('BOOL_TRUE') and len(response) == 1:
children = CIRelationManager.get_children(response[0]['_id'], ret_key=ret_key) # one floor
response[0].update(children)
current_app.logger.debug("search time is: {0}".format(time.time() - start))
return self.jsonify(numfound=numfound,
total=total,
page=page,
@ -159,6 +173,9 @@ class CISearchView(APIView):
counter=counter,
result=response)
def post(self):
return self.get()
class CIUnique(APIView):
url_prefix = "/ci/<int:ci_id>/unique"
@ -204,7 +221,7 @@ class CIHeartbeatView(APIView):
class CIFlushView(APIView):
url_prefix = ("/ci/flush", "/ci/<int:ci_id>/flush")
@auth_abandoned
# @auth_abandoned
def get(self, ci_id=None):
from api.tasks.cmdb import ci_cache
from api.lib.cmdb.const import CMDB_QUEUE
@ -214,4 +231,12 @@ class CIFlushView(APIView):
cis = CI.get_by(to_dict=False)
for ci in cis:
ci_cache.apply_async([ci.id], queue=CMDB_QUEUE)
return self.jsonify(code=200)
class CIAutoDiscoveryStatisticsView(APIView):
url_prefix = "/ci/adc/statistics"
def get(self):
return self.jsonify(CIManager.get_ad_statistics())

26
cmdb-api/api/views/cmdb/ci_relation.py
View File

@ -9,6 +9,7 @@ from flask import request
from api.lib.cmdb.cache import RelationTypeCache
from api.lib.cmdb.ci import CIRelationManager
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.cmdb.search import SearchError
from api.lib.cmdb.search.ci_relation.search import Search
from api.lib.decorator import args_required
@ -22,7 +23,6 @@ from api.resource import APIView
class CIRelationSearchView(APIView):
url_prefix = ("/ci_relations/s", "/ci_relations/search")
@auth_abandoned
def get(self):
"""@params: q: query statement
fl: filter by column
@ -31,7 +31,6 @@ class CIRelationSearchView(APIView):
level: default is 1
facet: statistic
"""
page = get_page(request.values.get("page", 1))
count = get_page_size(request.values.get("count") or request.values.get("page_size"))
@ -42,9 +41,10 @@ class CIRelationSearchView(APIView):
fl = handle_arg_list(request.values.get('fl', ""))
facet = handle_arg_list(request.values.get("facet", ""))
sort = request.values.get("sort")
reverse = request.values.get("reverse") in current_app.config.get('BOOL_TRUE')
start = time.time()
s = Search(root_id, level, query, fl, facet, page, count, sort)
s = Search(root_id, level, query, fl, facet, page, count, sort, reverse)
try:
response, counter, total, page, numfound, facet = s.search()
except SearchError as e:
@ -89,7 +89,7 @@ class GetSecondCIsView(APIView):
try:
relation_type_id = RelationTypeCache.get(relation_type).id if relation_type else None
except AttributeError:
return abort(400, "invalid relation type <{0}>".format(relation_type))
return abort(400, ErrFormat.invalid_relation_type.format(relation_type))
manager = CIRelationManager()
numfound, total, second_cis = manager.get_second_cis(
@ -147,12 +147,12 @@ class BatchCreateOrUpdateCIRelationView(APIView):
url_prefix = "/ci_relations/batch"
@args_required('ci_ids')
@args_required('parents')
def post(self):
ci_ids = request.values.get('ci_ids')
parents = request.values.get('parents')
ci_ids = list(map(int, request.values.get('ci_ids')))
parents = list(map(int, request.values.get('parents', [])))
children = list(map(int, request.values.get('children', [])))
CIRelationManager.batch_update(ci_ids, parents)
CIRelationManager.batch_update(ci_ids, parents, children)
return self.jsonify(code=200)
@ -160,3 +160,13 @@ class BatchCreateOrUpdateCIRelationView(APIView):
@args_required('parents')
def put(self):
return self.post()
@args_required('ci_ids')
@args_required('parents')
def delete(self):
ci_ids = list(map(int, request.values.get('ci_ids')))
parents = list(map(int, request.values.get('parents', [])))
CIRelationManager.batch_delete(ci_ids, parents)
return self.jsonify(code=200)

322
cmdb-api/api/views/cmdb/ci_type.py
View File

@ -1,9 +1,13 @@
# -*- coding:utf-8 -*-
import json
from io import BytesIO
from flask import abort
from flask import current_app
from flask import request
from flask import session
from api.lib.cmdb.cache import AttributeCache
from api.lib.cmdb.cache import CITypeCache
@ -11,9 +15,23 @@ from api.lib.cmdb.ci_type import CITypeAttributeGroupManager
from api.lib.cmdb.ci_type import CITypeAttributeManager
from api.lib.cmdb.ci_type import CITypeGroupManager
from api.lib.cmdb.ci_type import CITypeManager
from api.lib.cmdb.const import RoleEnum
from api.lib.cmdb.ci_type import CITypeTemplateManager
from api.lib.cmdb.ci_type import CITypeTriggerManager
from api.lib.cmdb.ci_type import CITypeUniqueConstraintManager
from api.lib.cmdb.const import PermEnum, ResourceTypeEnum, RoleEnum
from api.lib.cmdb.perms import CIFilterPermsCRUD
from api.lib.cmdb.preference import PreferenceManager
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.decorator import args_required
from api.lib.decorator import args_validate
from api.lib.perm.acl.acl import ACLManager
from api.lib.perm.acl.acl import has_perm_from_args
from api.lib.perm.acl.acl import is_app_admin
from api.lib.perm.acl.acl import role_required
from api.lib.perm.acl.cache import AppCache
from api.lib.perm.acl.role import RoleCRUD
from api.lib.perm.acl.role import RoleRelationCRUD
from api.lib.perm.auth import auth_with_app_token
from api.lib.utils import handle_arg_list
from api.resource import APIView
@ -34,8 +52,8 @@ class CITypeView(APIView):
return self.jsonify(numfound=count, ci_types=ci_types)
@role_required(RoleEnum.CONFIG)
@args_required("name")
@args_validate(CITypeManager.cls)
def post(self):
params = request.values
@ -49,44 +67,67 @@ class CITypeView(APIView):
return self.jsonify(type_id=type_id)
@role_required(RoleEnum.CONFIG)
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_validate(CITypeManager.cls)
def put(self, type_id):
params = request.values
manager = CITypeManager()
manager.update(type_id, **params)
return self.jsonify(type_id=type_id)
@role_required(RoleEnum.CONFIG)
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
def delete(self, type_id):
CITypeManager.delete(type_id)
return self.jsonify(type_id=type_id)
class CITypeGroupView(APIView):
url_prefix = ("/ci_types/groups", "/ci_types/groups/<int:gid>")
url_prefix = ("/ci_types/groups",
"/ci_types/groups/config",
"/ci_types/groups/order",
"/ci_types/groups/<int:gid>")
def get(self):
config_required = True if "/config" in request.url else False
need_other = request.values.get("need_other")
return self.jsonify(CITypeGroupManager.get(need_other))
return self.jsonify(CITypeGroupManager.get(need_other, config_required))
@role_required(RoleEnum.CONFIG)
@args_required("name")
@args_validate(CITypeGroupManager.cls)
def post(self):
name = request.values.get("name")
group = CITypeGroupManager.add(name)
return self.jsonify(group.to_dict())
@role_required(RoleEnum.CONFIG)
def put(self, gid):
name = request.values.get('name')
@args_validate(CITypeGroupManager.cls)
def put(self, gid=None):
if "/order" in request.url:
if RoleEnum.CONFIG not in session.get("acl", {}).get("parentRoles", []) and not is_app_admin("cmdb"):
return abort(403, ErrFormat.role_required.format(RoleEnum.CONFIG))
group_ids = request.values.get('group_ids')
CITypeGroupManager.order(group_ids)
return self.jsonify(group_ids=group_ids)
name = request.values.get('name') or abort(400, ErrFormat.argument_value_required.format("name"))
type_ids = request.values.get('type_ids')
CITypeGroupManager.update(gid, name, type_ids)
return self.jsonify(gid=gid)
@role_required(RoleEnum.CONFIG)
def delete(self, gid):
CITypeGroupManager.delete(gid)
type_ids = request.values.get("type_ids")
CITypeGroupManager.delete(gid, type_ids)
return self.jsonify(gid=gid)
@ -97,16 +138,18 @@ class CITypeQueryView(APIView):
def get(self):
q = request.args.get("q")
res = CITypeManager.query(q)
return self.jsonify(ci_type=res)
class EnableCITypeView(APIView):
url_prefix = "/ci_types/<int:type_id>/enable"
@role_required(RoleEnum.CONFIG)
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
def post(self, type_id):
enable = request.values.get("enable", True)
CITypeManager.set_enabled(type_id, enabled=enable)
return self.jsonify(type_id=type_id, enable=enable)
@ -114,16 +157,22 @@ class CITypeAttributeView(APIView):
url_prefix = ("/ci_types/<int:type_id>/attributes", "/ci_types/<string:type_name>/attributes")
def get(self, type_id=None, type_name=None):
t = CITypeCache.get(type_id) or CITypeCache.get(type_name) or abort(404, "CIType does not exist")
t = CITypeCache.get(type_id) or CITypeCache.get(type_name) or abort(404, ErrFormat.ci_type_not_found)
type_id = t.id
unique_id = t.unique_id
unique = AttributeCache.get(unique_id).name
return self.jsonify(attributes=CITypeAttributeManager.get_attributes_by_type_id(type_id),
attr_filter = CIFilterPermsCRUD.get_attr_filter(type_id)
attributes = CITypeAttributeManager.get_attributes_by_type_id(type_id)
if attr_filter:
attributes = [i for i in attributes if i['name'] in attr_filter]
return self.jsonify(attributes=attributes,
type_id=type_id,
unique_id=unique_id,
unique=unique)
@role_required(RoleEnum.CONFIG)
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_required("attr_id")
def post(self, type_id=None):
attr_id_list = handle_arg_list(request.values.get("attr_id"))
@ -131,31 +180,34 @@ class CITypeAttributeView(APIView):
params.pop("attr_id", "")
CITypeAttributeManager.add(type_id, attr_id_list, **params)
return self.jsonify(attributes=attr_id_list)
@role_required(RoleEnum.CONFIG)
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_required("attributes")
def put(self, type_id=None):
"""
attributes is list, only support raw data request
:param type_id:
:return:
:param type_id:
:return:
"""
attributes = request.values.get("attributes")
current_app.logger.debug(attributes)
if not isinstance(attributes, list):
return abort(400, "attributes must be list")
return abort(400, ErrFormat.argument_attributes_must_be_list)
CITypeAttributeManager.update(type_id, attributes)
return self.jsonify(attributes=attributes)
@role_required(RoleEnum.CONFIG)
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_required("attr_id")
def delete(self, type_id=None):
"""
Form request: attr_id is a string, separated by commas
Raw data request: attr_id is a list
:param type_id:
:return:
:param type_id:
:return:
"""
attr_id_list = handle_arg_list(request.values.get("attr_id", ""))
@ -164,6 +216,25 @@ class CITypeAttributeView(APIView):
return self.jsonify(attributes=attr_id_list)
class CITypesAttributeView(APIView):
url_prefix = ("/ci_types/attributes",)
@args_required("type_ids", value_required=True)
def get(self):
type_ids = handle_arg_list(request.values.get('type_ids'))
attr_names = set()
attributes = list()
for type_id in type_ids:
_attributes = CITypeAttributeManager.get_attributes_by_type_id(type_id)
for _attr in _attributes:
if _attr['name'] not in attr_names:
attr_names.add(_attr['name'])
attributes.append(_attr)
return self.jsonify(attributes=attributes)
class CITypeAttributeTransferView(APIView):
url_prefix = "/ci_types/<int:type_id>/attributes/transfer"
@ -198,10 +269,18 @@ class CITypeAttributeGroupView(APIView):
def get(self, type_id):
need_other = request.values.get("need_other")
return self.jsonify(CITypeAttributeGroupManager.get_by_type_id(type_id, need_other))
groups = CITypeAttributeGroupManager.get_by_type_id(type_id, need_other)
@role_required(RoleEnum.CONFIG)
attr_filter = CIFilterPermsCRUD.get_attr_filter(type_id)
if attr_filter:
for group in groups:
group['attributes'] = [attr for attr in (group.get('attributes') or []) if attr['name'] in attr_filter]
return self.jsonify(groups)
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_required("name")
@args_validate(CITypeAttributeGroupManager.cls)
def post(self, type_id):
name = request.values.get("name").strip()
order = request.values.get("order") or 0
@ -213,7 +292,9 @@ class CITypeAttributeGroupView(APIView):
current_app.logger.warning(group.id)
return self.jsonify(group_id=group.id)
@role_required(RoleEnum.CONFIG)
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_required("name")
@args_validate(CITypeAttributeGroupManager.cls)
def put(self, group_id):
name = request.values.get("name")
order = request.values.get("order") or 0
@ -224,7 +305,198 @@ class CITypeAttributeGroupView(APIView):
CITypeAttributeGroupManager.update(group_id, name, attr_order, order)
return self.jsonify(group_id=group_id)
@role_required(RoleEnum.CONFIG)
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
def delete(self, group_id):
CITypeAttributeGroupManager.delete(group_id)
return self.jsonify(group_id=group_id)
class CITypeTemplateView(APIView):
url_prefix = ("/ci_types/template/import", "/ci_types/template/export")
@role_required(RoleEnum.CONFIG)
def get(self): # export
return self.jsonify(
dict(ci_type_template=CITypeTemplateManager.export_template()))
@role_required(RoleEnum.CONFIG)
def post(self): # import
tpt = request.values.get('ci_type_template') or {}
CITypeTemplateManager().import_template(tpt)
return self.jsonify(code=200)
class CITypeCanDefineComputed(APIView):
url_prefix = "/ci_types/can_define_computed"
@role_required(PermEnum.CONFIG)
def get(self):
return self.jsonify(code=200)
class CITypeTemplateFileView(APIView):
url_prefix = ("/ci_types/template/import/file", "/ci_types/template/export/file")
@role_required(RoleEnum.CONFIG)
def get(self): # export
tpt_json = CITypeTemplateManager.export_template()
tpt_json = dict(ci_type_template=tpt_json)
bf = BytesIO()
bf.write(bytes(json.dumps(tpt_json).encode('utf-8')))
bf.seek(0)
return self.send_file(bf,
as_attachment=True,
attachment_filename="cmdb_template.json",
mimetype='application/json',
cache_timeout=0)
@role_required(RoleEnum.CONFIG)
def post(self): # import
f = request.files.get('file')
if f is None:
return abort(400, ErrFormat.argument_file_not_found)
content = f.read()
try:
content = json.loads(content)
except:
return abort(400, ErrFormat.invalid_json)
tpt = content.get('ci_type_template')
CITypeTemplateManager().import_template(tpt)
return self.jsonify(code=200)
class CITypeUniqueConstraintView(APIView):
url_prefix = ("/ci_types/<int:type_id>/unique_constraint", "/ci_types/<int:type_id>/unique_constraint/<int:_id>")
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
def get(self, type_id):
return self.jsonify(CITypeUniqueConstraintManager.get_detail(type_id))
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_required("attr_ids")
def post(self, type_id):
attr_ids = request.values.get('attr_ids')
return self.jsonify(CITypeUniqueConstraintManager().add(type_id, attr_ids))
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_required("attr_ids")
def put(self, type_id, _id):
assert type_id is not None
attr_ids = request.values.get('attr_ids')
return self.jsonify(CITypeUniqueConstraintManager().update(_id, attr_ids))
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
def delete(self, type_id, _id):
assert type_id is not None
CITypeUniqueConstraintManager().delete(_id)
return self.jsonify(code=200)
class CITypeTriggerView(APIView):
url_prefix = ("/ci_types/<int:type_id>/triggers", "/ci_types/<int:type_id>/triggers/<int:_id>")
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
def get(self, type_id):
return self.jsonify(CITypeTriggerManager.get(type_id))
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_required("attr_id")
@args_required("notify")
def post(self, type_id):
attr_id = request.values.get('attr_id')
notify = request.values.get('notify')
return self.jsonify(CITypeTriggerManager().add(type_id, attr_id, notify))
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_required("notify")
def put(self, type_id, _id):
assert type_id is not None
notify = request.values.get('notify')
return self.jsonify(CITypeTriggerManager().update(_id, notify))
@has_perm_from_args("type_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
def delete(self, type_id, _id):
assert type_id is not None
CITypeTriggerManager().delete(_id)
return self.jsonify(code=200)
class CITypeGrantView(APIView):
url_prefix = "/ci_types/<int:type_id>/roles/<int:rid>/grant"
def post(self, type_id, rid):
perms = request.values.pop('perms', None)
if request.values.get('attr_filter'):
request.values['attr_filter'] = handle_arg_list(request.values.get('attr_filter', ''))
_type = CITypeCache.get(type_id)
type_name = _type and _type.name or abort(404, ErrFormat.ci_type_not_found)
acl = ACLManager('cmdb')
if not acl.has_permission(type_name, ResourceTypeEnum.CI_TYPE, PermEnum.GRANT) and \
not is_app_admin('cmdb'):
return abort(403, ErrFormat.no_permission.format(type_name, PermEnum.GRANT))
acl.grant_resource_to_role_by_rid(type_name, rid, ResourceTypeEnum.CI_TYPE, perms)
CIFilterPermsCRUD().add(type_id=type_id, rid=rid, **request.values)
return self.jsonify(code=200)
class CITypeRevokeView(APIView):
url_prefix = "/ci_types/<int:type_id>/roles/<int:rid>/revoke"
@args_required('perms')
def post(self, type_id, rid):
perms = request.values.pop('perms', None)
if request.values.get('attr_filter'):
request.values['attr_filter'] = handle_arg_list(request.values.get('attr_filter', ''))
_type = CITypeCache.get(type_id)
type_name = _type and _type.name or abort(404, ErrFormat.ci_type_not_found)
acl = ACLManager('cmdb')
if not acl.has_permission(type_name, ResourceTypeEnum.CI_TYPE, PermEnum.GRANT) and \
not is_app_admin('cmdb'):
return abort(403, ErrFormat.no_permission.format(type_name, PermEnum.GRANT))
acl.revoke_resource_from_role_by_rid(type_name, rid, ResourceTypeEnum.CI_TYPE, perms)
if PermEnum.READ in perms:
CIFilterPermsCRUD().delete(type_id=type_id, rid=rid)
app_id = AppCache.get('cmdb').id
users = RoleRelationCRUD.get_users_by_rid(rid, app_id)
for i in (users or []):
if i.get('role', {}).get('id') and not RoleCRUD.has_permission(
i.get('role').get('id'), type_name, ResourceTypeEnum.CI_TYPE, app_id, PermEnum.READ):
PreferenceManager.delete_by_type_id(type_id, i.get('uid'))
return self.jsonify(type_id=type_id, rid=rid)
class CITypeFilterPermissionView(APIView):
url_prefix = "/ci_types/<int:type_id>/filters/permissions"
@auth_with_app_token
def get(self, type_id):
return self.jsonify(CIFilterPermsCRUD().get(type_id))

54
cmdb-api/api/views/cmdb/ci_type_relation.py
View File

@ -1,11 +1,17 @@
# -*- coding:utf-8 -*-
from flask import abort
from flask import request
from api.lib.cmdb.ci_type import CITypeManager
from api.lib.cmdb.ci_type import CITypeRelationManager
from api.lib.cmdb.const import RoleEnum
from api.lib.cmdb.const import PermEnum, ResourceTypeEnum, RoleEnum
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.decorator import args_required
from api.lib.perm.acl.acl import ACLManager
from api.lib.perm.acl.acl import has_perm_from_args
from api.lib.perm.acl.acl import is_app_admin
from api.lib.perm.acl.acl import role_required
from api.resource import APIView
@ -33,15 +39,16 @@ class CITypeRelationView(APIView):
return self.jsonify(res)
@role_required(RoleEnum.CONFIG)
@has_perm_from_args("parent_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
@args_required("relation_type_id")
def post(self, parent_id, child_id):
relation_type_id = request.values.get("relation_type_id")
ctr_id = CITypeRelationManager.add(parent_id, child_id, relation_type_id)
constraint = request.values.get("constraint")
ctr_id = CITypeRelationManager.add(parent_id, child_id, relation_type_id, constraint)
return self.jsonify(ctr_id=ctr_id)
@role_required(RoleEnum.CONFIG)
@has_perm_from_args("parent_id", ResourceTypeEnum.CI, PermEnum.CONFIG, CITypeManager.get_name_by_id)
def delete(self, parent_id, child_id):
CITypeRelationManager.delete_2(parent_id, child_id)
@ -56,3 +63,42 @@ class CITypeRelationDelete2View(APIView):
CITypeRelationManager.delete(ctr_id)
return self.jsonify(code=200, ctr_id=ctr_id)
class CITypeRelationGrantView(APIView):
url_prefix = "/ci_type_relations/<int:parent_id>/<int:child_id>/roles/<int:rid>/grant"
def post(self, parent_id, child_id, rid):
p = CITypeManager.check_is_existed(parent_id)
c = CITypeManager.check_is_existed(child_id)
resource_name = CITypeRelationManager.acl_resource_name(p.name, c.name)
perms = request.values.get('perms')
acl = ACLManager('cmdb')
if not acl.has_permission(resource_name, ResourceTypeEnum.CI_TYPE_RELATION, PermEnum.GRANT) and \
not is_app_admin('cmdb'):
return abort(403, ErrFormat.no_permission.format(resource_name, PermEnum.GRANT))
acl.grant_resource_to_role_by_rid(resource_name, rid, ResourceTypeEnum.CI_TYPE_RELATION, perms)
return self.jsonify(code=200)
class CITypeRelationRevokeView(APIView):
url_prefix = "/ci_type_relations/<int:parent_id>/<int:child_id>/roles/<int:rid>/revoke"
def post(self, parent_id, child_id, rid):
p = CITypeManager.check_is_existed(parent_id)
c = CITypeManager.check_is_existed(child_id)
resource_name = CITypeRelationManager.acl_resource_name(p.name, c.name)
perms = request.values.get('perms')
acl = ACLManager('cmdb')
if not acl.has_permission(resource_name, ResourceTypeEnum.CI_TYPE_RELATION, PermEnum.GRANT) and \
not is_app_admin('cmdb'):
return abort(403, ErrFormat.no_permission.format(resource_name, PermEnum.GRANT))
acl.revoke_resource_from_role_by_rid(resource_name, rid, ResourceTypeEnum.CI_TYPE_RELATION, perms)
return self.jsonify(code=200)

72
cmdb-api/api/views/cmdb/custom_dashboard.py Normal file
View File

@ -0,0 +1,72 @@
# -*- coding:utf-8 -*-
from flask import request
from api.lib.cmdb.const import RoleEnum
from api.lib.cmdb.custom_dashboard import CustomDashboardManager
from api.lib.cmdb.custom_dashboard import SystemConfigManager
from api.lib.decorator import args_required
from api.lib.decorator import args_validate
from api.lib.perm.acl.acl import role_required
from api.resource import APIView
class CustomDashboardApiView(APIView):
url_prefix = ("/custom_dashboard", "/custom_dashboard/<int:_id>", "/custom_dashboard/batch")
def get(self):
return self.jsonify(CustomDashboardManager.get())
@role_required(RoleEnum.CONFIG)
@args_validate(CustomDashboardManager.cls)
def post(self):
cm = CustomDashboardManager.add(**request.values)
return self.jsonify(cm.to_dict())
@role_required(RoleEnum.CONFIG)
@args_validate(CustomDashboardManager.cls)
def put(self, _id=None):
if _id is not None:
cm = CustomDashboardManager.update(_id, **request.values)
return self.jsonify(cm.to_dict())
CustomDashboardManager.batch_update(request.values.get("id2options"))
return self.jsonify(id2options=request.values.get('id2options'))
@role_required(RoleEnum.CONFIG)
def delete(self, _id):
CustomDashboardManager.delete(_id)
return self.jsonify(code=200)
class SystemConfigApiView(APIView):
url_prefix = ("/system_config",)
@role_required(RoleEnum.CONFIG)
@args_required("name", value_required=True)
def get(self):
return self.jsonify(SystemConfigManager.get(request.values['name']))
@role_required(RoleEnum.CONFIG)
@args_validate(SystemConfigManager.cls)
@args_required("name", value_required=True)
@args_required("option", value_required=True)
def post(self):
cm = SystemConfigManager.create_or_update(**request.values)
return self.jsonify(cm.to_dict())
def put(self, _id=None):
return self.post()
@role_required(RoleEnum.CONFIG)
@args_required("name")
def delete(self):
CustomDashboardManager.delete(request.values['name'])
return self.jsonify(code=200)

72
cmdb-api/api/views/cmdb/history.py
View File

@ -6,58 +6,92 @@ import datetime
from flask import abort
from flask import request
from api.lib.cmdb.ci import CIManager
from api.lib.cmdb.const import ResourceTypeEnum, PermEnum
from api.lib.cmdb.const import RoleEnum
from api.lib.cmdb.history import AttributeHistoryManger
from api.lib.cmdb.history import CITypeHistoryManager
from api.lib.cmdb.resp_format import ErrFormat
from api.lib.perm.acl.acl import has_perm_from_args
from api.lib.perm.acl.acl import role_required
from api.lib.utils import get_page
from api.lib.utils import get_page_size
from api.resource import APIView
class RecordView(APIView):
url_prefix = "/history/records"
url_prefix = ("/history/records/attribute", "/history/records/relation")
@role_required(RoleEnum.CONFIG)
def get(self):
page = get_page(request.values.get("page", 1))
page_size = get_page_size(request.values.get("page_size"))
_start = request.values.get("start")
_end = request.values.get("end")
username = request.values.get("username", "")
operate_type = request.values.get("operate_type", "")
type_id = request.values.get("type_id")
start, end = None, None
if _start:
try:
start = datetime.datetime.strptime(_start, '%Y-%m-%d %H:%M:%S')
except ValueError:
abort(400, 'incorrect start date time')
return abort(400, ErrFormat.datetime_argument_invalid.format('start'))
if _end:
try:
end = datetime.datetime.strptime(_end, '%Y-%m-%d %H:%M:%S')
except ValueError:
abort(400, 'incorrect end date time')
return abort(400, ErrFormat.datetime_argument_invalid.format('start'))
numfound, total, res = AttributeHistoryManger.get_records(start, end, username, page, page_size)
if "attribute" in request.url:
total, res = AttributeHistoryManger.get_records_for_attributes(start, end, username, page, page_size,
operate_type,
type_id,
request.values.get('ci_id'),
request.values.get('attr_id'))
return self.jsonify(records=res,
total=total,
**request.values)
else:
total, res, cis = AttributeHistoryManger.get_records_for_relation(start, end, username, page, page_size,
operate_type,
type_id,
request.values.get('first_ci_id'),
request.values.get('second_ci_id'))
return self.jsonify(numfound=numfound,
records=res,
page=page,
total=total,
start=_start,
end=_end,
username=username)
return self.jsonify(records=res,
total=total,
cis=cis,
**request.values)
class CIHistoryView(APIView):
url_prefix = "/history/ci/<int:ci_id>"
@has_perm_from_args("ci_id", ResourceTypeEnum.CI, PermEnum.READ, CIManager.get_type_name)
def get(self, ci_id):
result = AttributeHistoryManger.get_by_ci_id(ci_id)
return self.jsonify(result)
class RecordDetailView(APIView):
url_prefix = "/history/records/<int:record_id>"
class CITypeHistoryView(APIView):
url_prefix = "/history/ci_types"
def get(self, record_id):
username, timestamp, attr_dict, rel_dict = AttributeHistoryManger.get_record_detail(record_id)
return self.jsonify(username=username,
timestamp=timestamp,
attr_history=attr_dict,
rel_history=rel_dict)
@role_required(RoleEnum.CONFIG)
def get(self):
type_id = request.values.get("type_id")
username = request.values.get("username")
operate_type = request.values.get("operate_type")
page = get_page(request.values.get('page', 1))
page_size = get_page_size(request.values.get('page_size', 1))
numfound, result = CITypeHistoryManager.get(page, page_size, username,
type_id=type_id, operate_type=operate_type)
return self.jsonify(page=page,
page_size=page_size,
numfound=numfound,
total=len(result),
result=result)

Some files were not shown because too many files have changed in this diff Show More